Review Form

Reviewer Persona: The Guardian (Adversarial Skeptic)

---

Summary

This paper introduces "AnA," an autonomous driving system architecture designed to improve efficiency and safety by making the perception module "attentive." The core idea is to establish a query-based interface between the planning and perception modules. The planner, using its knowledge of the driving context, requests focused perception tasks (e.g., high-accuracy localization of specific agents), allowing the system to dynamically allocate computational resources. The authors claim this approach significantly reduces collisions and compute usage compared to traditional, non-attentive pipelines.

While the concept of a feedback loop from planning to perception is sound, this paper suffers from significant methodological weaknesses, overstated claims, and an evaluation that fails to rigorously substantiate its core contributions. The evidence presented does not adequately support the headline claims of performance improvement, and the novelty of some technical components is questionable.

---

Strengths

1. Problem Formulation: The paper correctly identifies a critical issue in modern ADS: the inefficiency of running expensive perception algorithms uniformly across the entire sensory input, regardless of the immediate driving context.
2. Architectural Concept: The high-level architectural proposal of a query-based, feedback-driven pipeline between planning and perception is a sensible and promising research direction.

---

Weaknesses

1. Grossly Overstated Performance Claims: The abstract and introduction make bold quantitative claims that are not supported by a holistic view of the presented data.

   • "reducing collisions by 3x" (Abstract): This claim is highly misleading. My analysis of Table 4 (p. 41) shows that this "3x" figure appears to be derived from a cherry-picked comparison in low-speed scenarios where the absolute number of collisions is low to begin with. For instance, in Scenario 4 at 12 m/s, Ours has "NC" (0 collisions) while the 2P-Heuristic baseline has 7.84. This is not a "3x reduction." In more challenging, high-speed scenarios (e.g., S1 at 24 m/s), the reduction is a modest 12% (11.81 vs. 10.41). Averaged across all high-risk scenarios, the improvement is far from the advertised 3x.
   • "reduces compute usage by 44%" (Abstract): This claim is based exclusively on the performance in low-risk, low-stress scenarios. Figure 7 (p. 41) shows this 44% reduction for Scenario S6. However, the authors themselves state that in high-risk situations, AnA "increases the ingestion rate" (Section 6.3.1, p. 41). The paper provides no data on GPU utilization for the more critical high-risk scenarios (S1-S5). It is likely that in these situations, where AnA queries for more refined processing, the computational savings diminish or disappear entirely. The claim is therefore not representative of the system's overall performance.

2. Weak and Potentially Unfair Baselines: The experimental comparison is fundamentally flawed. AnA is a system with a planner-to-perception feedback loop. The baselines (2P-All, 2P-Moving, 2P-Heuristic) are open-loop perception-only heuristics. The observed performance gain may simply be due to the existence of any feedback loop, rather than the specific query mechanism proposed by AnA. A more rigorous evaluation would have included a baseline with a simpler feedback mechanism to isolate the contribution of AnA's specific design. Furthermore, the 2P-All baseline, which refines every single detected object, is a strawman argument; no practical system would be designed this way.

3. Questionable Novelty of Technical Components:

   • The "RoI localization" mechanism described in Section 4.2.2 and Listing 1 (p. 38) appears to be a standard, first-order motion model (i.e., new_position = old_position + velocity * delta_time). This is a basic prediction step, common in any tracking algorithm (e.g., the prediction step of a Kalman filter), and framing it as a novel contribution is a significant overstatement.
   • The exception handling mechanism (Query Monitor, Section 4.3) is described in a vague, hand-wavy manner. It is unclear what the planner concretely does when an exception is raised, or how the "higher-level vision operator" is implemented. This critical component for ensuring safety is not sufficiently detailed or evaluated.

4. Contradictory System Description: The motivation argues against processing "all agents [...] all the time" (Section 1, p. 34). However, the AnA architecture still relies on a "first pass" (Section 4.2.1) that runs a detector on every single frame to generate initial detections. This "standing query" contradicts the core premise of targeted attention. The system does not avoid processing the entire scene; it merely adds a second, selective stage. The efficiency gains are thus more limited than the introduction implies.

5. Evaluation in a Non-Adversarial, Simulated Environment: The entire evaluation is conducted in the CARLA simulator. While useful for prototyping, simulators often fail to capture the long tail of real-world sensor noise, lighting conditions, and unpredictable agent behaviors. The paper makes claims about "adversarial events" but the scenarios (Table 2, p. 39) seem to be standard, scripted traffic situations. There is no evidence of a truly adversarial evaluation designed to find failure modes of the attention mechanism (e.g., a suddenly appearing, occluded pedestrian that the "standing query" might miss). Furthermore, the evaluation is performed on a high-end RTX 3090, which is not representative of resource-constrained automotive hardware, where the overhead of the AnA framework itself could become a significant factor.

---

Questions to Address In Rebuttal

1. Please provide a table showing the average collision reduction percentage for Ours vs. the 2P-Heuristic baseline, calculated across all high-risk scenarios (S1-S5) and all speeds. How does this averaged data support the "3x reduction" claim made in the abstract?
2. Please provide GPU utilization graphs, analogous to Figure 7, for the high-risk scenarios (e.g., S1, S5). Does the 44% computational saving hold when the system is under stress and must issue more refinement queries?
3. The primary difference between your method and the baselines is the planner-perception feedback loop. How can you justify that the observed benefits stem from your specific query interface design, and not merely from the presence of a feedback loop in general? Why was a simpler feedback-based baseline not included for comparison?
4. Can you clarify the concrete implementation of the exception handling mechanism? Specifically, what actions does the planner take when it receives an exception from the Query Monitor, and what is the "higher-level vision operator" mentioned in Section 4.3?
5. Please clarify the novelty of the RoI localization method (Listing 1) in relation to standard state prediction techniques used in object tracking, such as the prediction step in a Kalman filter.
6. Given that the system's safety hinges on the initial "standing query," what is the system's behavior if a critical agent is missed in this first pass (e.g., due to the choice of detection threshold mentioned in Section 3.2.1)? Has this failure mode been tested?

Reviewer: The Synthesizer (Contextual Analyst)

---

Summary

This paper presents AnA, an architectural redesign of the standard Autonomous Driving System (ADS) software stack. The authors identify a key inefficiency in current systems: the perception module operates largely "obliviously," processing all sensor data with maximum effort, irrespective of the current driving context or the specific needs of the downstream planning module.

The core contribution is to refactor this monolithic, feed-forward pipeline into a dynamic, feedback-driven system inspired by human cognition. AnA introduces a formal separation between low-cost, continuous "awareness" (achieved via standing queries) and high-cost, on-demand "attention" (achieved via ad-hoc queries). A novel query interface allows the planning module to explicitly request detailed perceptual information about specific objects or regions that are relevant to its decision-making process. This allows the system to dynamically allocate its computational resources to what matters most for safety and navigation. The authors demonstrate through simulation that their approach not only reduces computational load significantly (up to 44% GPU utilization reduction) but also improves driving safety, reducing collision severity and frequency in high-risk scenarios.

Strengths

1. Elegant and Principled Architectural Abstraction: The paper's most significant strength is its core idea. The explicit separation of "awareness" and "attention" is a powerful and intuitive abstraction that directly addresses a well-known but often poorly-articulated problem in ADS design. By drawing an analogy to human cognition (as mentioned in Section 1), the authors provide a strong conceptual foundation for their work. This moves beyond ad-hoc optimizations and proposes a new, more intelligent way to structure the entire perception-planning interaction. The introduction of the query interface is the key mechanism that makes this abstraction concrete and implementable.

2. Solves a Critical Cross-Cutting Problem: This work is not just another incremental improvement to a specific model; it tackles a fundamental systems-level issue at the intersection of computer vision, robotics, and real-time systems. The "oblivious perception" problem is a major bottleneck for deploying powerful AI models on resource-constrained edge hardware. AnA's success in simultaneously improving safety metrics and computational efficiency is a powerful result, demonstrating a way out of the common trade-off where higher safety requires more compute.

3. Broad Potential for Impact: While framed in the context of autonomous driving, the core concept of a query-driven perception system is highly generalizable. This architectural pattern could be influential in other domains of robotics (e.g., manipulation, drone navigation) where agents must perceive and act in complex, dynamic environments under computational constraints. It provides a formal "language" for downstream modules to communicate their information needs to upstream sensor processing modules, a long-standing challenge in robotics system integration.

4. Strong Empirical Validation: The evaluation in Section 6 is thorough. The authors compare their system against a well-chosen set of baselines that represent different points in the design space (e.g., single-pass vs. two-pass, heuristic vs. all-object refinement). The results, particularly the improved driving scores in high-speed and complex scenarios (Table 4) combined with the drastic GPU reduction (Figure 7), make a compelling case for the proposed architecture.

Weaknesses

While the core idea is excellent, its current realization and evaluation have some limitations that are worth noting. These should be viewed not as fatal flaws, but as important avenues for future work.

1. The "Black Swan" Problem: The entire system's safety is predicated on the initial, low-cost "awareness" pass (the standing query) successfully detecting potential threats. An agent that is completely missed in this first stage will never trigger a high-fidelity "attention" query. While the paper uses a high-recall detector, the risk of a false negative on a fast-approaching, out-of-distribution object remains. The paper does not deeply explore the ultimate safety net for this failure mode.

2. Simplicity of the Current Query System: The query types described (e.g., refining a bounding box, estimating speed) are foundational but represent a fraction of the information a planner might need. The paper does not fully explore the scalability of this interface. For instance, in a chaotic urban intersection with dozens of agents, how does the query executor prioritize and manage a potential flood of ad-hoc queries? What happens when queries conflict or when the system is saturated?

3. Evaluation in Simulation: The use of the CARLA simulator is a standard and necessary step, but it abstracts away many real-world complexities. The robustness of the RoI re-localization algorithm (Section 4.2, Figure 5), for example, might be challenged by real-world phenomena like severe sensor noise, motion blur, or unpredictable ego-vehicle odometry errors. The bridge from these promising simulation results to a physically deployed system remains a significant undertaking.

Questions to Address In Rebuttal

1. Could you elaborate on the system's robustness to catastrophic failures in the initial "awareness" stage? If a high-speed vehicle is missed by the initial standing query due to, for example, adverse weather or it being an unusual object class, is there any fallback mechanism, or does the system remain blind to it until it's too late?

2. The paper focuses on a single-camera setup for clarity. How do you envision the query interface and executor scaling to a full sensor suite with multiple cameras, LiDAR, and RADAR? Would a query be directed at a specific sensor, or would it be an abstract query for an "agent," leaving the executor to decide how to best fuse information to satisfy it?

3. In extremely cluttered scenarios (e.g., a crowded city square), the planner might deem a large number of agents "relevant," potentially overwhelming the system with ad-hoc queries and negating the computational savings. Have you explored the system's behavior at these high-load extremes, and are there mechanisms for graceful degradation or query prioritization?

Review Form: The Innovator

---

Summary

The paper presents "AnA," an "attentive" autonomous driving system. The core thesis is that the traditional, strictly feed-forward autonomous driving software (ADS) pipeline (Perception -> Prediction -> Planning) is inefficient and suboptimal. It processes all sensor data with maximum effort, irrespective of the driving context or the vehicle's immediate plans.

To address this, the authors propose refactoring this pipeline to include a feedback loop. Specifically, the planning module can issue "queries" back to the perception module to request specific information. This creates a dichotomy: a low-cost, continuous "awareness" mode for general scene understanding, and a high-cost, on-demand "attention" mode that directs perception resources to agents and regions relevant to the ego-vehicle's planned trajectory. The proposed system is composed of three primary components: a query interface, a query executor, and a query monitor. The authors claim this new architecture improves safety in high-risk scenarios while significantly reducing computational load (and thus energy consumption) in low-risk scenarios.

---

Strengths

From a novelty perspective, the primary strength of this work lies in its specific architectural contribution. The authors have correctly identified a well-known inefficiency in modular ADS stacks and have proposed a concrete, engineered solution.

1. Formalization of a Planner-to-Perception Feedback Loop: The central novel idea is the formalization of top-down, goal-directed processing within a classic modular ADS. While the abstract concept of "active vision" or "top-down attention" is decades old in robotics and computer vision, its instantiation as a formal Query Interface between the planning and perception modules in a modern ADS stack is a novel systems contribution. This moves beyond ad-hoc heuristics and proposes a principled software abstraction.

2. The "Awareness vs. Attention" Dichotomy: The explicit separation of perception into a baseline "awareness" scan and a targeted "attention" query (Section 2.3, page 35) is a clean and powerful implementation of the core idea. This is a well-established pattern in cognitive science and other areas of computer science, but its application as a guiding principle for refactoring an entire ADS pipeline is a notable contribution.

3. Specific Query Mechanism: The paper details a specific mechanism where the planner communicates its needs in terms of its future trajectory g_ego, a latency budget T_q, and an error bound E_q (Section 3.2.2, page 36). This elevates the idea beyond a simple "look here" command to a more expressive, performance-aware contract between system modules. This level of detail in the interface design is a novel aspect of the work.

---

Weaknesses

The paper's primary weakness, from a novelty standpoint, is that the foundational concept of dynamic resource allocation for perception is not new. The authors' contribution is an excellent piece of systems engineering and integration, but the originality of the underlying principle is limited.

1. Overlap with Prior Art on Dynamic Perception: The idea of selectively processing parts of a scene or using different algorithms based on context has been explored before. The authors themselves cite REMIX [22], which partitions frames to run different vision algorithms on different regions of interest. While AnA's mechanism is different—driven explicitly by the planner's intent rather than more generic saliency—the fundamental goal of optimizing the perception stack is shared. The delta lies in the source of the optimization signal (planner vs. a more general context), which is an important but incremental, rather than revolutionary, step.

2. Limited Acknowledgment of "Active Vision" Lineage: The work exists within a long history of "active vision" in robotics, where a robot's planned actions guide its sensing strategy. A more thorough discussion of how AnA's specific architectural choices advance this long-standing paradigm would help to better situate the paper's novelty. Without this context, the claims of novelty may seem stronger than they are to a reader not deeply familiar with the robotics literature.

3. Complexity vs. Novelty: The proposed solution introduces significant new complexity: a query language, a scheduler (the executor), and a monitoring system. The performance gains appear to justify this complexity. However, the novelty is in the combination and application of these known systems components (interfaces, schedulers) to the ADS domain, not in the invention of fundamentally new algorithms. The contribution is architectural, not algorithmic.

---

Questions to Address In Rebuttal

1. The concept of dynamically allocating perception resources is present in prior work such as REMIX [22], which also partitions frames to apply different vision algorithms. Could the authors please clarify the key conceptual delta between their planner-driven query system and the region-based approach in REMIX? Is the primary novelty the source of the signal (i.e., the planner's future trajectory) or the query-based mechanism itself?

2. The proposed query interface and executor add a non-trivial layer of complexity to the ADS pipeline. Have the authors considered the failure modes of this new abstraction? For example, in a dense urban scenario, what prevents the planner from issuing a large number of ad-hoc queries, effectively overwhelming the executor and negating the system's efficiency gains by forcing it into a constant high-attention state?

3. The paper frames the novelty around the system architecture. How does the expressiveness of the query language itself factor into this novelty? The current queries seem focused on refining the location/class of known objects. How does the AnA framework extend to more semantic queries that a future planner might need, such as, "confirm all cross-traffic has stopped" or "determine if the pedestrian intends to cross"? Is the architectural novelty robust to the inclusion of more complex, non-object-centric queries?

Paper Title: AnyKey: A Key-Value SSD for All Workload Types Reviewer: The Guardian (Adversarial Skeptic)

---

Summary

This paper identifies a performance vulnerability in state-of-the-art LSM-tree-based Key-Value SSDs, such as PinK, when subjected to workloads with low value-to-key (low-v/k) ratios. The authors argue that as the relative key size increases, the metadata required to index KV pairs bloats, exceeding the capacity of the device's internal DRAM and forcing metadata to be stored on flash. This results in additional, high-latency flash accesses for read operations, severely degrading performance.

To address this, the authors propose AnyKey, a novel KV-SSD design. AnyKey's core idea is to manage KV pairs in "data segment groups" and reduce metadata by storing only a single key and a list of partial hashes for each group. This allows the metadata to fit within the DRAM even for low-v/k workloads. The design also incorporates a "value log" to separate values from keys, aiming to reduce write amplification during LSM-tree compaction. An enhanced version, AnyKey+, introduces a modified compaction policy to mitigate "compaction chains." The evaluation, conducted using the FEMU emulator, shows that AnyKey and AnyKey+ significantly outperform PinK on the targeted low-v/k workloads and remain competitive on high-v/k workloads.

Strengths

1. Clear and Well-Motivated Problem: The paper's primary contribution is its identification and empirical validation of a significant performance cliff in existing KV-SSD designs. The analysis in Section 3.2 (page 5) and the motivating data in Figure 2 (page 2) compellingly demonstrate that the metadata management in PinK is not robust to changes in key-value size distributions, specifically for low-v/k workloads. This is a valuable and timely observation.

2. Intuitive Core Design Idea: The fundamental approach of grouping KV pairs to amortize metadata overhead is logical. By creating data segment groups and using hashes to locate items within them, the authors present a plausible mechanism for shrinking the total metadata footprint.

Weaknesses

My primary concerns with this paper relate to overstated claims, questionable design choices in the enhanced system, and an evaluation that does not sufficiently support the paper's central thesis.

1. Overstated and Unsupported Generality: The title claims the design is for "All Workload Types." This is a strong claim that the evaluation fails to substantiate. The experiments in Section 5 (page 10) are conducted almost exclusively with a 20% write ratio. The behavior of an LSM-tree-based system is critically dependent on the write rate, as this drives compactions. A write-heavy workload (e.g., 50% or 80% writes) would stress the value log and trigger compactions far more frequently. Without evaluating such scenarios, the claim of suitability for "all" workloads is unsubstantiated speculation.

2. Flawed Heuristic in AnyKey+: The proposed enhancement in AnyKey+ to prevent "compaction chains" (Section 4.5.2, page 10) is deeply problematic. The authors propose that during a log-triggered compaction, once the destination level reaches a certain threshold, the remaining values are written back into the value log. This appears to defeat the primary purpose of a log-triggered compaction, which is to reclaim space in the value log. This design choice feels like an ad-hoc patch that addresses a symptom rather than the root cause. A more principled approach would involve a more sophisticated compaction scheduling policy, rather than a mechanism that knowingly writes data back into the very space it is trying to clean. The logic here is circular and requires much stronger justification.

3. Insufficient Analysis of System Overheads:

   • Computational Overhead: The design introduces significant computational work (hashing, sorting based on hashes) onto the SSD's internal controller, which is a notoriously resource-constrained environment. While the authors cite a 79ns latency for a single hash operation (Section 4.4.2, page 9), they fail to analyze the aggregate impact on the controller's CPU utilization during intensive operations like a major compaction of millions of keys. It is plausible that the controller becomes a bottleneck, a factor that an emulator like FEMU may not fully capture compared to real hardware.
   • Hash Collisions: The authors claim a hash collision rate of only 0.075% (Section 4.1.1, page 6) but do not state how this figure was derived. Is this a theoretical probability, an average across all test workloads, or a worst-case measurement? The mechanism to handle collisions (reading adjacent pages) introduces unpredictable latency spikes, and the true impact of this cannot be assessed without more information.
   • Read Latency from Value Log: The use of a value log, inspired by WiscKey, is known to introduce a potential "double read" penalty: one read for the key/pointer in the LSM-tree and a second, separate read for the value in the log. The paper completely fails to discuss or quantify the impact of this on read latency. While AnyKey may reduce flash accesses from metadata spills, it is unclear if it simply trades them for extra flash accesses to the value log.

4. Questionable Scalability Claims: In Section 5.9 (page 13), the authors claim scalability by extrapolating the metadata size for a hypothetical 4TB SSD. This is a simple calculation, not an experimental result. A 4TB SSD would have a proportionally larger DRAM (e.g., 4GB). With a 4GB DRAM, it is entirely possible that the original problem of metadata bloat in PinK would be significantly reduced or eliminated for the tested workloads, thus weakening the motivation for AnyKey at larger scales. The claim of superior scalability is therefore not proven.

Questions to Address In Rebuttal

1. Please justify the title's claim of suitability for "All Workload Types" given that the evaluation is limited to a single 20% write ratio. How would AnyKey perform under a 50% or 80% write-intensive workload?

2. The core mechanic of AnyKey+ for preventing compaction chains (writing values back into the value log, Section 4.5.2) seems counter-productive to the goal of reclaiming log space. Please provide a detailed justification for this design choice over other alternatives, such as implementing a more advanced compaction scheduling algorithm. An ablation study would be necessary here.

3. The computational overhead on the SSD controller is a critical factor for a real-world implementation. Can you provide data on the projected CPU utilization of the controller under AnyKey, especially during major compactions? Furthermore, how was the 0.075% hash collision rate (Section 4.1.1) determined?

4. What is the performance penalty for reads that must access the value log (the "double read" problem)? Please quantify how frequently this occurs across your workloads and what its impact is on both average and 95th percentile read latency.

5. Your scalability argument in Section 5.9 is based on extrapolation. A 4TB SSD would likely have a ~4GB DRAM. Could you demonstrate that PinK's metadata would still exceed this 4GB DRAM for the Crypto1 workload, or would the problem be mitigated, thus reducing the benefit of AnyKey at that scale?

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper identifies and addresses a critical performance limitation in current state-of-the-art Key-Value SSD (KV-SSD) designs. The authors observe that existing LSM-tree-based KV-SSDs, such as PinK, are implicitly optimized for workloads with a high value-to-key (high-v/k) ratio, where values are significantly larger than keys. They demonstrate that for an important and previously under-explored class of "low-v/k" workloads, the metadata required to index the KV pairs grows too large to fit in the device's limited internal DRAM. This overflow leads to excessive flash reads for metadata, causing a severe degradation in latency and IOPS.

The authors propose AnyKey, a novel KV-SSD architecture designed to be robust across all workload types. The core contribution is a new metadata management scheme that groups KV pairs into "data segment groups" and uses hashes to create compact, group-level metadata. This approach dramatically reduces the overall metadata size, ensuring it remains resident in DRAM even for low-v/k workloads. To manage the write amplification that this grouping would otherwise cause during compactions, AnyKey cleverly adapts the key-value separation principle (popularized by systems like WiscKey) by introducing an on-device "value log." The paper's comprehensive evaluation, using the FEMU emulator, validates that AnyKey significantly outperforms the state-of-the-art on low-v/k workloads while remaining competitive on traditional high-v/k workloads.

Strengths

1. Excellent Problem Formulation and Motivation: The paper's primary strength is its clear and compelling identification of a real-world problem. The authors do not create a strawman; instead, they conduct a survey of various applications—from databases and caches to blockchains—to demonstrate the prevalence of low-v/k workloads (Figure 1, page 2). The subsequent analysis showing the performance cliff of a state-of-the-art design on these workloads (Figure 2, page 2) provides a powerful and unambiguous motivation for the work.

2. Insightful Diagnosis of the Root Cause: The re-evaluation of PinK in Section 3 is exemplary. The authors precisely diagnose the problem as metadata bloat leading to DRAM overflow, rather than some other less fundamental issue. This clear diagnosis directly informs their solution and makes the logic of their design easy to follow.

3. Elegant Synthesis of Existing Concepts: The design of AnyKey is a masterful synthesis of powerful ideas from the storage literature, adapted for the unique, resource-constrained environment of an SSD controller.

   • The core idea of grouping data to amortize metadata overhead is simple but highly effective.
   • The use of hashes to facilitate intra-group lookups is a classic space-saving technique applied intelligently here.
   • Most importantly, the adaptation of key-value separation (from WiscKey [49]) into an on-device value log is a critical insight. This demonstrates a deep understanding of the LSM-tree write amplification problem and how to mitigate the consequences of their own design choices (i.e., moving whole groups of values during compaction would be too expensive).

4. Broadening the Viability of KV-SSDs: This work significantly strengthens the overall case for KV-SSDs. By addressing a major performance pathology, AnyKey makes the technology viable for a much wider range of applications. It helps move KV-SSDs from being a niche solution for specific data patterns (large values) to a more general-purpose, high-performance storage primitive. This has a potentially high impact on the field.

Weaknesses

While the paper is strong, its positioning and discussion could be enhanced by exploring the following points:

1. Understated Connection to Host-Level Optimizations: The paper cites WiscKey [49] as the inspiration for its value log. However, it misses an opportunity to frame its contribution more broadly. WiscKey demonstrated the power of key-value separation on the host. AnyKey's core contribution can be seen as successfully translating this powerful host-level architectural pattern into the much more constrained device-level environment. Discussing the unique challenges of this translation (e.g., limited compute for garbage collecting the value log, managing DRAM for pointers vs. host RAM) would better contextualize the work and highlight its novelty.

2. The "One Size Fits All" Implication: The title "A Key-Value SSD for All Workload Types" is ambitious. The proposed design is static and is clearly superior for low-v/k workloads. However, for extreme high-v/k workloads, the overhead of storing hashes and managing the value log might be slightly less optimal than a purely value-size-tuned system. The paper lacks a discussion of adaptivity. Could a KV-SSD dynamically choose between a PinK-style metadata scheme and an AnyKey-style scheme based on workload characteristics? Acknowledging that AnyKey represents a more robust point in the design space, rather than a single final answer for all possible workloads, would add nuance.

3. The Data Segment Group as a Tuning Parameter: The "data segment group" is the fundamental unit in AnyKey's design. The authors fix its size in their implementation (32 pages, Section 4.1.1, page 6), but the performance trade-offs associated with this parameter are not fully explored. A smaller group size would lead to more metadata (trending towards PinK), while a larger group size would reduce metadata but could increase read amplification for point lookups (reading a larger group) and hash collision probability. A brief discussion on the sensitivity to this parameter would be valuable.

Questions to Address In Rebuttal

1. Could the authors elaborate on the challenges of implementing a key-value separation model (i.e., the value log and its garbage collection) within the resource constraints of an SSD controller, as opposed to a host-based system like WiscKey which can leverage more abundant CPU and memory resources?

2. The title claims the design is for "All Workload Types." Could you discuss the performance of AnyKey in a truly mixed workload scenario, where requests for both high-v/k and low-v/k data are interleaved? Does the design offer a robust average case, or would it be possible to construct an adversarial workload that challenges the static design choices (e.g., the value log size)?

3. The data segment group size seems to be a critical tuning parameter. Could you comment on the trade-offs involved in selecting this size? How would performance (e.g., read latency, metadata size, compaction overhead) be affected if this parameter were significantly smaller or larger?

Review Form: The Innovator (Novelty Specialist)

Summary

This paper identifies a performance bottleneck in existing LSM-tree-based Key-Value SSDs (KV-SSDs), exemplified by PinK, when subjected to workloads with low value-to-key (low-v/k) ratios. The authors correctly diagnose the problem as metadata bloat: when keys are large, the metadata required to index them consumes excessive device-internal DRAM, forcing metadata to be paged out to flash and incurring significant read amplification.

To address this, the authors propose "AnyKey," a novel KV-SSD design. The core contribution is a new metadata and data layout scheme. Instead of indexing individual key-value pairs, AnyKey groups KV pairs into "data segment groups" (spanning multiple physical pages) and indexes the group. Crucially, within these groups, KV pairs are sorted by a fixed-size hash of the key, not the key itself. This allows the metadata to remain compact regardless of key size. The design also incorporates a value log, inspired by prior work, to optimize compactions. An enhanced version, AnyKey+, further refines the compaction process to handle high-v/k workloads more gracefully.

Strengths

The primary strength of this paper is the identification and solution of a genuine problem with a conceptually novel mechanism. While individual components of the solution have appeared in other contexts, their synthesis into this specific architecture for KV-SSDs is new.

1. Core Novel Idea: The central novel contribution is the structure of the "data segment group" combined with its corresponding metadata in the "level list." Specifically, the decision to sort KV entities within a group by a fixed-size hash value is a clever method to decouple the metadata size from the key size. This directly and effectively addresses the identified problem of metadata bloat for low-v/k workloads. This is a distinct departure from traditional LSM-tree designs that sort SSTables purely by key.

2. Problem-Driven Innovation: The proposed architecture is not an arbitrary new design but is purpose-built to solve the well-motivated problem of large keys. The authors provide a clear causal link between the problem (metadata bloat) and their solution (hash-sorted groups with compact metadata).

Weaknesses

While the core idea is novel, the paper's overall novelty is diluted by its heavy reliance on integrating well-known, pre-existing techniques from the literature. A more explicit acknowledgment and differentiation would strengthen the paper.

1. Direct Adoption of Prior Art: The most significant component borrowed from prior art is the value log. As the authors state in Section 4.1.3 (page 7), this mechanism is "inspired by [49]". This is an understatement; the technique of separating keys from values into a log to reduce compaction write amplification is the central contribution of WiscKey [49]. While applying it in a KV-SSD is a valid engineering step, it is not a novel concept.

2. Re-implementation of Standard Concepts: The "hash list" described in Section 4.1.2 (page 7) serves as a filter to avoid unnecessary flash reads for a data segment group. This is functionally equivalent to the Bloom filters used ubiquitously in nearly all production LSM-tree implementations (e.g., RocksDB, LevelDB). The authors' choice to use a perfect list of hashes instead of a probabilistic data structure like a Bloom filter is an implementation detail and an engineering trade-off (space vs. computational complexity), not a novel architectural concept.

3. Complexity vs. Benefit Trade-off: The novel mechanism of sorting by hash introduces significant secondary complexities that a traditional key-sorted LSM-tree does not have.

   • Hash Collisions: The design must now handle hash collisions, requiring extra metadata ("hash collision bits," Figure 7, page 6) and potentially extra page reads.
   • Broken Key Ordering: The fundamental benefit of an LSM-tree—maintaining key-sorted data for efficient range scans—is broken by this design. The authors' proposed workaround in Section 4.2.5 (page 8), which involves storing an extra index of key locations within the first page of a group, feels like an ad-hoc patch. This re-introduces a form of key-dependent metadata, undermining the design's primary goal, and likely adds non-trivial storage and computation overhead for range queries.

Questions to Address In Rebuttal

1. Delineating Novelty: Could the authors more clearly articulate the "delta" between their work and prior art? Beyond acknowledging WiscKey [49], please contrast the proposed hash-based grouping and sorting with other hash-partitioned or hash-indexed LSM-tree designs. What makes sorting within a run by hash fundamentally different and novel compared to partitioning an LSM-tree by hash at a higher level?

2. Justification of Complexity: The proposed architecture is substantially more complex than the baseline PinK design. It introduces hash generation, collision handling, two separate compaction triggers (tree- and log-based), and a special mechanism for range queries. Have the authors considered simpler alternatives to mitigate metadata bloat, such as applying key-prefix compression or other general-purpose compression schemes to the metadata (level lists and meta segments) in PinK? A justification for why such simpler methods would be insufficient would strengthen the case for this more complex architecture.

3. Quantifying the Range Query Overhead: Sorting by hash is a critical design choice with a major drawback for range queries. The paper claims in Section 5.7 that performance improves for longer scans but provides limited data. Could the authors provide a quantitative analysis of the storage overhead for the range query index (mentioned in Section 4.2.5) and a direct performance comparison against PinK for range queries of varying lengths (e.g., scanning 10, 100, and 1000 keys)? PinK’s key-sorted data layout should give it a significant structural advantage here, and it is crucial to understand the magnitude of the trade-off AnyKey makes.

Reviewer: The Guardian

---

Summary

The paper identifies that the gradient computation phase in rasterization-based differentiable rendering workloads, such as 3D Gaussian Splatting, is severely bottlenecked by atomic operations. The authors make two key observations: (1) threads within a warp exhibit high spatial locality, frequently targeting the same memory address for atomic updates, and (2) due to control divergence, only a subset of threads in a warp may be active at any given time. To address this, they propose ARC (Adaptive Atomic Reduction), a primitive that performs warp-level reduction within the Streaming Multiprocessor (SM) to reduce traffic to the L2 atomic units (ROPs). ARC adaptively schedules reductions between the SM core and the L2 ROPs based on contention. The paper presents two implementations: a hardware proposal (ARC-HW) evaluated in simulation, and a software-only version (ARC-SW) evaluated on real hardware.

While the paper identifies a valid and important bottleneck, its proposed software solution (ARC-SW) relies on a manually-tuned hyperparameter that undermines its claims of being "adaptive" and robust. Furthermore, the hardware evaluation (ARC-HW) is confined to simulation against baselines that may not fully represent the design space, making it difficult to assess its real-world viability and advantage over existing techniques.

Strengths

1. Timely Workload Characterization: The paper provides a valuable and detailed performance analysis of an emerging and important class of workloads (raster-based differentiable rendering). Identifying the atomic bottleneck in the gradient computation step (Section 3, pages 4-5) is a solid contribution to the community.
2. Sound Core Idea: The fundamental concept of leveraging the high intra-warp locality (Observation 1, Section 3.1) for reduction at the SM core is logical and well-motivated. This is a classic optimization pattern, and its application here is appropriate.
3. Comprehensive Proposal: The authors present both a hardware (ARC-HW) and a software (ARC-SW) implementation, demonstrating a thorough exploration of the solution space.

Weaknesses

1. The "Balancing Threshold" is a Critical Flaw in the Software Approach: The entire adaptive mechanism of ARC-SW hinges on a balance_thr hyperparameter (Section 4.4, page 8 and Section 5.5, page 9). The authors admit this threshold "needs to be tuned for each workload." Figure 23 (page 12) demonstrates this fragility perfectly: an incorrect threshold choice leads to significant performance degradation, even resulting in a slowdown compared to the baseline for NV and PS workloads.

   • The proposed auto-tuning mechanism (Section 5.5.3, page 10) is a patch, not a solution. It automates the search for a static value but does not make the system dynamically adaptive to runtime phase changes. What if the optimal threshold changes as the model converges or as different parts of the scene are rendered? The paper provides no evidence of the robustness of this static, periodically-tuned value. This reliance on a workload-and-system-specific magic number severely limits the practicality and generality of ARC-SW.

2. Limited and Potentially Unfair Evaluation Baselines:

   • Hardware (Simulation): The comparison against LAB [32] and PHI [78] is only performed in simulation. LAB-ideal, which assumes a dedicated, contention-free SRAM, is an unrealistic upper bound and serves to inflate ARC-HW's relative performance. While ARC-HW outperforms the more realistic LAB implementation, the performance gap is smaller. The lack of a real hardware comparison makes it impossible to know if simulation artifacts are influencing the results.
   • Software (Real Hardware): The paper compares ARC-SW against a baseline atomicAdd and the NVIDIA CCCL library. The authors state that "significant engineering efforts were needed to make CCCL work correctly for these workloads" (Section 7.2, page 13). This claim requires significant substantiation. What specific aspects of CCCL failed or were inefficient? CCCL primitives are highly optimized. If the issue is CCCL's assumption of full warp participation, this is precisely the problem ARC is meant to solve and should be the basis of a much deeper, more principled comparison, rather than a dismissal based on implementation difficulty. Without this detail, the comparison feels incomplete.

3. Overstated Generality of Observations: The paper's core motivation rests on Observation 1: "Threads within a warp are likely to update the same parameters." In Section 3.1 (page 5), the authors make the strong claim that for workload 3D-PL, "over 99% of warps have all their threads update the same memory location." This is a single data point. This claim must be substantiated with data across all evaluated workloads to be considered a general characteristic. Without this, the entire premise may only apply to a subset of scenarios.

4. Unaddressed Overheads and Assumptions:

   • The butterfly reduction in SW-B requires inactive threads to be re-activated to generate zero-value updates (Section 5.5.2, page 10). This introduces redundant computation and instruction overhead, which is not quantified.
   • The paper makes the standard but important simplification that floating-point additions are commutative (Section 5.2, page 8). While acceptable for many ML workloads, this is a numerical precision issue that should be acknowledged as a potential source of non-determinism and divergence from the baseline.
   • The overhead of the auto-tuning process itself (running one iteration with 32 different thresholds) is claimed to be "negligible" (Section 5.5.3, page 10) but is not measured.

Questions to Address In Rebuttal

1. Please justify how ARC-SW can be considered "adaptive" when its performance is critically dependent on a balance_thr hyperparameter that must be profiled and tuned for each specific workload and dataset. How does your proposed auto-tuner handle dynamic phase behavior during a single training run where the optimal threshold might change?
2. Can you provide a more detailed technical explanation for why the highly-optimized CCCL library was a poor fit for these workloads? What specific primitives were used, and what were their fundamental limitations that ARC-SW overcomes?
3. Please provide quantitative data supporting "Observation 1" (the high degree of intra-warp address locality for atomics) across all workloads listed in Table 2, not just the single 3D-PL example.
4. Regarding the ARC-SW-B implementation, what is the measured instruction and execution overhead of forcing inactive threads to perform zero-value updates, especially in warps with low thread activity? How does this overhead impact the choice of the balance_thr?

Paper Title: ARC: Warp-level Adaptive Atomic Reduction in GPUs to Accelerate Differentiable Rendering Reviewer Persona: The Synthesizer (Contextual Analyst)

---

Summary

This paper identifies a critical performance bottleneck in the training phase of modern, rasterization-based differentiable rendering techniques like 3D Gaussian Splatting (3DGS). The authors profile these workloads and find that the gradient computation step, which relies heavily on atomic operations to accumulate gradients, consumes over 50% of the training time and is limited by contention at the L2 cache's atomic units (ROPs).

The core contribution is ARC, a new primitive designed to alleviate this bottleneck based on two key workload characteristics: (1) extremely high intra-warp locality, where most or all threads in a warp atomically update the same memory address, and (2) dynamic and variable thread activity within a warp due to control divergence. ARC proposes a two-pronged solution: first, it performs warp-level reduction directly within the SM core using registers, drastically reducing the number of atomic requests sent to the L2. Second, it adaptively distributes the atomic computation, sending high-contention updates (many threads per warp) to the local SM reduction unit and low-contention updates to the traditional L2 ROPs. The authors present two implementations: ARC-HW, a low-overhead hardware proposal, and ARC-SW, a practical and immediately applicable software-only library. The work demonstrates significant speedups on both real and simulated hardware, effectively mitigating a major performance limiter in an important and rapidly evolving application domain.

Strengths

1. Timely and High-Impact Problem Identification: The paper does an excellent job of positioning itself at the intersection of computer architecture and a cutting-edge application domain. While the graphics and ML communities have celebrated the rendering speed of 3DGS, this work is one of the first to perform a deep architectural analysis of its training pipeline and identify the next major bottleneck. The profiling results in Section 3 (page 4-5) are clear and compelling, establishing that the problem is both real and significant. This is a classic example of strong systems research: finding and solving a new problem created by advances in other fields.

2. Insightful Workload Characterization: The strength of the proposed solution stems directly from the authors' keen observations about the workload. The identification of near-perfect intra-warp address locality for atomics (Observation 1, page 5) is the critical insight that makes warp-level reduction so effective. Equally important is the characterization of control divergence leading to partially active warps (Observation 2, page 5), which correctly dismisses naive warp-level primitives and motivates the "adaptive" nature of ARC. This demonstrates a deep understanding of the application's behavior.

3. Elegant and Well-Reasoned Solution: The ARC primitive is an elegant solution that directly maps to the identified problem characteristics. Instead of proposing a complex, general-purpose atomic accelerator, it leverages existing GPU structures (warp schedulers, registers, address coalescing units) with minimal additions. The idea of dynamically balancing the load between the SM core and the L2 ROPs is particularly clever, as it prevents the new reduction unit from becoming a bottleneck itself and ensures that the system's full atomic throughput is utilized.

4. Comprehensive and Practical Evaluation: The authors' two-pronged implementation strategy (ARC-HW and ARC-SW) is a major strength. ARC-HW demonstrates the full potential of the idea in a future architecture, while the open-sourced ARC-SW provides a pathway for immediate impact on current hardware. The evaluation is thorough, using multiple workloads, real GPUs (NVIDIA 4090/3060), and simulation. The comparisons against relevant prior work (LAB, PHI in Section 7.1, page 11) and state-of-the-art software libraries (CCCL in Section 7.2, page 13) convincingly show the superiority of their targeted approach for this specific workload class.

Weaknesses

While this is a strong paper, there are opportunities to further contextualize the contribution.

1. Limited Discussion on Broader Applicability: The paper is rightly focused on differentiable rendering, as it's a strong motivating application. However, its conclusion as a "general atomic primitive" (Section 9, page 14) could be better supported. The authors correctly note in Section 5.6 (page 10) that workloads like graph analytics do not benefit due to low intra-warp locality. This is a crucial point of contrast. The paper would be strengthened by a more speculative discussion on what other emerging application domains might exhibit this high-locality, high-contention atomic pattern. Are there kernels in scientific computing, physics simulation, or other ML models (e.g., certain types of mixture-of-experts or sparse models) where this primitive could be equally transformative?

2. Practicality of Software Tuning: The ARC-SW implementation relies on a "balancing threshold" that must be tuned for optimal performance. The authors propose a pragmatic auto-tuning approach (Section 5.5.3, page 10), but this still introduces a layer of complexity for the developer. The sensitivity analysis in Figure 23 (page 12) shows that a poor choice can negate the benefits or even cause slowdowns. This is less a fundamental flaw and more a practical consideration that could be discussed further, perhaps in the context of creating more robust heuristics that don't require per-workload profiling.

3. Hardware Implementation Nuances: The area overhead calculation for ARC-HW (Section 5.4, page 9) is appreciated and suggests the proposal is lightweight. However, for an architecture conference, a brief discussion on other potential hardware complexities would be welcome. For instance, would the new reduction unit and its scheduler introduce any new pipeline hazards or affect the timing of the SM's front-end? How does the new atomred instruction interact with the memory consistency model beyond commutativity? These are second-order effects but would add depth to the hardware proposal.

Questions to Address In Rebuttal

1. Generality and Future Workloads: Beyond the well-chosen domain of differentiable rendering, can the authors speculate on other emerging application domains or computational patterns that exhibit the high intra-warp atomic locality necessary for ARC to be effective? This would help frame ARC as a forward-looking architectural feature rather than a point solution for a single application class.

2. Robustness of the Adaptive Scheduling: The auto-tuning approach for the balancing threshold in ARC-SW is practical. Could the authors comment on how this threshold might vary with future GPU architectures that have different SM-to-ROP unit ratios? Could a more robust, non-profile-based heuristic be developed, perhaps by dynamically monitoring LSU queue length at runtime?

3. Path to Adoption via Compilers: The programmer burden of manually inserting ARC-SW calls or atomred instructions is a practical barrier to adoption. Could the authors envision a path for compiler toolchains to automatically detect the high-locality atomic reduction pattern within loops and perform the necessary code transformations, thereby making ARC's benefits accessible without manual intervention?

Review Persona: The Innovator

---

Summary

The paper identifies a performance bottleneck in modern rasterization-based differentiable rendering workloads, specifically the high volume of atomic operations during the gradient computation step. The authors make two key observations about these workloads: (1) high intra-warp locality, where most threads in a warp atomically update the same memory location, and (2) high variance in the number of active threads per warp performing these atomics.

To address this, they propose ARC, a primitive for warp-level adaptive atomic reduction. The core claims to novelty rest on two ideas: 1. Performing warp-level reduction directly at the GPU sub-core using registers, bypassing the L1 cache/LSU path used by prior atomic aggregation techniques. 2. An adaptive scheduling mechanism that dynamically distributes atomic operations between this new sub-core reduction path and the traditional L2 atomic units (ROPs), based on contention.

The authors present both a hardware (ARC-HW) and a software-only (ARC-SW) implementation.

---

Strengths (from a Novelty Perspective)

1. Novel Architectural Synthesis: The primary novel contribution is not warp-level reduction in isolation, but the specific architectural synthesis of performing this reduction in registers at the sub-core level combined with a dynamic, contention-aware scheduling policy. While prior art has explored atomic aggregation, it has focused on memory structures like the L1 cache or shared memory SRAM (e.g., PHI [78], LAB [32]). The authors' insight that these approaches are still bottlenecked at the LSU in this specific workload (Section 3.2, page 5) and that a register-based reduction can bypass this is a novel and valuable architectural observation.

2. Adaptive Scheduling Mechanism: The concept of dynamically arbitrating atomic workloads between the SM and the L2 ROPs is a novel element. Prior works generally propose a monolithic mechanism (e.g., always buffer in L1). ARC’s proposal to use SM-level reduction for high-contention warps and L2 ROPs for low-contention warps is a new approach to maximizing total atomic throughput across the chip. The proposed hardware implementation, which uses LDST unit stalls as a proxy for ROP contention (Section 4.3, page 7), is an elegant and low-overhead scheduler design.

3. Hardware Support for Divergence: The proposed hardware implementation (ARC-HW) presents a novel approach to handling thread divergence within a reduction operation. By leveraging the existing address coalescing unit to generate a thread mask for threads updating the same location (Section 5.1, page 8), it efficiently handles the irregularity (Observation 2) that the authors identify, without the software overhead of explicit masking and conditional logic. This is a significant advancement over software libraries that often require all threads to be active.

---

Weaknesses (from a Novelty Perspective)

1. Overstated Novelty of Underlying Primitives: The paper's core ideas are built upon concepts that are not, in themselves, new. Warp-level reduction using shuffle instructions has been a standard optimization technique in GPU programming for nearly a decade, and is the basis for libraries like NVIDIA's CUB [15] and CCCL [14], which the authors compare against. The software implementation, ARC-SW, relies entirely on existing primitives like __shfl and __match (Section 5.5, page 9). The novelty is therefore not in the reduction algorithm itself, but purely in its adaptive application.

2. Existing Software Patterns for Divergence: The serialized reduction pattern used in ARC-SW-S to handle divergence (Figure 15, page 10) is a well-known parallel programming pattern. A skilled programmer could construct a similar mechanism using __match to identify active lanes and then loop within a leader thread. The contribution is thus one of engineering and packaging this pattern with an adaptive heuristic, rather than inventing a fundamentally new method for handling divergence in software.

3. Incremental Advancement in Scheduling: While the adaptive scheduling is a strength, the software implementation relies on a simple, tunable threshold (balance_thr). This is a common heuristic-based approach. The novelty is in applying this heuristic to arbitrate between two different hardware paths for atomics, which is clever, but the mechanism itself is not a breakthrough in scheduling theory.

---

Questions to Address In Rebuttal

1. Clarification of Delta vs. Prior Art (LAB/PHI): The primary difference claimed over LAB [32] and PHI [78] is the location of aggregation (registers vs. L1/SRAM). Could LAB or PHI be modified to specifically exploit intra-warp locality, for instance, by coalescing updates from a single warp before writing to their respective SRAM/cache buffers? Please clarify why a register-based approach is fundamentally different and not just an alternative implementation of the same core idea of on-SM aggregation.

2. Necessity of Adaptive Threshold: The adaptive threshold in ARC-SW is a key component. How does this compare against a simpler, non-adaptive policy? For example, a policy where warp-level reduction is only used if all 32 threads in the warp are active and update the same address (a condition easily checked with __match), falling back to standard atomics otherwise. This would isolate the performance gain of the adaptivity itself from the gain of simply using warp-shuffle where possible.

3. Novelty of Software Divergence Handling: The authors claim that libraries like CCCL require all threads in a warp to be active. While this is true for their most efficient primitives, it is possible for a programmer to manually implement a divergent-safe reduction using __match and a leader thread, as is done in ARC-SW-S. Could the authors please clarify if ARC-SW-S provides a fundamentally new capability, or if it primarily offers a convenient and well-optimized implementation of an existing, albeit complex, programming pattern?

Reviewer Persona: The Guardian (Adversarial Skeptic)

---

Summary

The authors present Apophenia, a system designed to automate the process of trace identification in implicitly parallel, task-based runtime systems. The work is motivated by the brittleness and complexity of manual trace annotations, which hinder program compositionality and require deep system knowledge. The proposed solution reframes the problem of trace discovery as an online string analysis problem. A stream of tasks, hashed into tokens, is analyzed to find repeated, non-overlapping substrings, which are then proposed as candidate traces. The core technical contribution is a heuristic, suffix-array-based algorithm for identifying these candidate traces from a buffer of recent task history. The system is implemented and evaluated within the Legion runtime system on a set of large-scale scientific and machine learning applications. The results demonstrate that Apophenia can achieve performance competitive with (0.92x-1.03x) and in some cases superior to that of manually traced or untraced versions of the applications.

Strengths

1. Strong Motivation: The paper does an excellent job motivating the problem. The motivating example in Section 2, which demonstrates how a seemingly straightforward loop annotation can fail due to the runtime's internal memory management, is a compelling and concrete illustration of the problem's subtlety and importance. It clearly establishes the need for a solution beyond simple syntactic analysis.

2. Substantial Evaluation: The experimental evaluation is conducted on real, complex, and large-scale applications (S3D, HTR, FlexFlow, etc.) running on modern supercomputers. The comparison against both untraced and manually-tuned versions provides a strong baseline and context for the results. This is not a "toy problem" evaluation and lends significant weight to the performance claims.

3. Formal Problem Definition: Section 3 provides a clear, concise optimization problem that formalizes the goal of "finding good traces." This provides a solid theoretical foundation against which the proposed heuristic solution can be understood, even if not formally compared.

Weaknesses

My primary concerns with this submission relate to the heuristic nature of the core algorithms, the justification for key design choices, and the potential overstatement of the system's generality.

1. Heuristic Algorithm without Bounded Sub-Optimality: The core of the trace-finding mechanism is Algorithm 2, which is acknowledged to be a greedy heuristic. It sorts candidates by length and greedily selects the longest non-overlapping ones. While this may work well in practice for the applications tested, the paper provides no analysis of its potential failure modes. The optimization problem in Section 3 seeks to maximize total coverage, but a greedy selection based on length is not guaranteed to achieve this. It is plausible that a set of shorter, more frequent traces could provide better coverage than a single long trace that prevents the selection of others. The paper lacks a discussion of pathological cases or an analysis of how far from optimal this greedy strategy might be.

2. Arbitrary Heuristics in Trace Selection: The trace replayer's scoring function, described in Section 4.3, is a collection of heuristics that lack rigorous justification. The use of a capped count, an exponential decay based on time since last appearance, and a "slight" increase in score for previously replayed traces introduces several magic numbers and parameters into the system. The paper provides no sensitivity analysis for these parameters, leaving the reader to wonder how critical their specific tuning is to the system's success. This feels more like ad-hoc engineering than a principled design.

3. Unexplained Performance Gap with Manual Tracing: The results show that Apophenia's performance can be as low as 0.92x that of manual tracing. The paper attributes this gap to manual annotations being able to "leverage application knowledge to select traces that have lower replay overhead" (Section 6.1, page 9). This is a critical point that is not elaborated upon. What, precisely, is this "application knowledge"? Is it related to phases of computation, communication boundaries, or something else? And why is Apophenia fundamentally incapable of learning or inferring this? Without a detailed explanation, this appears to be a significant limitation of the automated approach that is not adequately explored.

4. Generality Claims Undermined by Legion-Specific Assumptions: The authors claim the ideas in Apophenia are "directly applied to other task-based runtime systems." However, a key design decision—the lack of speculation (Section 5.2)—is explicitly justified by the specific performance characteristics of the Legion runtime (i.e., a very expensive analysis phase). In a different runtime where analysis is cheaper, waiting for an entire trace to be issued before beginning replay could introduce significant pipeline bubbles and performance degradation. The claim of generality is therefore not fully substantiated by the evidence and arguments presented.

5. Practicality of Warmup Time: The warmup times presented in Figure 9, particularly the 300 iterations required for CFD and TorchSWE, are substantial. While the authors correctly note that these applications run for many more iterations in production, this long warmup period could be a prohibitive cost for debugging, development, or production runs with smaller problem sizes or shorter total execution times. This practical limitation is understated in the discussion.

Questions to Address In Rebuttal

1. Regarding Algorithm 2: Can the authors provide a concrete, even if synthetic, example of a task stream where the greedy, length-based selection strategy leads to a significantly sub-optimal trace set (in terms of total coverage) compared to the optimal solution defined in Section 3? Specifically, how does it handle multiple, interleaved, shorter repeating patterns that might be "masked" by a slightly longer but less frequent pattern?

2. Regarding the performance gap with manual tracing: Please provide a specific example of the "application knowledge" that a programmer uses to achieve up to 8% better performance than Apophenia. What is the structural property of the task graph or application logic that Apophenia fails to capture in this case?

3. Regarding the distributed synchronization mechanism in Section 5.1: The method to ensure determinism by agreeing on a "count of processed operations" is described at a high level. Please provide more detail on this mechanism. Is this a blocking collective operation? What is its communication pattern and performance cost, especially at scale?

4. Regarding the scoring function in Section 4.3: Please provide a justification for the choice of an exponential decay for the appearance count. A sensitivity analysis showing how performance changes with different decay rates, replay biases, and appearance count caps would significantly strengthen the claim that this is a robust mechanism.

Reviewer Persona: The Synthesizer (Contextual Analyst)

---

Summary

This paper presents Apophenia, a system that automates the process of tracing in implicitly parallel, task-based runtime systems. The core problem is that while tracing (memoizing dependence analysis for repeated task sequences) is a critical optimization for performance, manually annotating these traces is brittle, error-prone, and fundamentally at odds with the principle of software composition.

The authors' central contribution is to reframe this optimization challenge as an online string analysis problem. Apophenia treats the stream of tasks issued by an application as a sequence of tokens, hashes them, and applies a novel, efficient algorithm based on suffix arrays to dynamically identify long, frequently-repeating subsequences. These identified sequences are then automatically converted into traces for the underlying runtime system (in this case, Legion) to memoize and replay.

The evaluation, conducted on large-scale scientific and machine learning applications on production supercomputers, is comprehensive. It demonstrates that Apophenia can achieve performance nearly identical to that of carefully hand-tuned manual traces, while also successfully tracing and accelerating complex, composed applications that were previously considered untraceable, yielding speedups of up to 2.82x. In essence, the paper presents a "Just-In-Time (JIT) compiler for dependence analysis," a powerful and elegant solution to a significant problem in task-based parallel computing.

---

Strengths

1. Elegant and Novel Problem Formulation: The most significant strength of this work is its conceptual leap. By viewing the dynamic task stream as a string to be mined for patterns, the authors connect the world of runtime systems optimization with decades of research in string processing algorithms. This reframing is not only clever but also highly effective, as it transforms a messy, application-specific annotation problem into a well-defined, automatable analysis.

2. Clear and Compelling Motivation: The paper does an excellent job of motivating the work. The cuPyNumeric example in Section 2 is particularly effective. It provides a concrete, non-trivial instance where the "obvious" manual tracing strategy fails due to the internal mechanics of a library, perfectly illustrating the brittleness that Apophenia aims to solve. This grounds the work in a real and pressing need for developers of composable software on these platforms.

3. Strong, Realistic Evaluation: The evaluation is a major highlight. The authors test their system not on toy benchmarks, but on production-grade applications like S3D, HTR, and FlexFlow, running at scale on leading supercomputers (Perlmutter and Eos). Comparing against both an untraced baseline and a manually-optimized one provides a clear picture of Apophenia's effectiveness. The ability to match manual performance is a high bar to clear, and the ability to significantly outperform the baseline on previously untraceable applications demonstrates the system's practical value.

4. High Potential Impact on Usability and Composability: This work has the potential to be an enabling technology. Apophenia removes a significant burden from the programmer, insulating them from the low-level performance details of the runtime. This lowers the barrier to entry and, more importantly, restores the promise of compositionality. Developers can build complex applications from independent modules without fear that their interaction will create an untraceable performance bottleneck. This directly addresses a tension that has existed in systems like Legion for years.

---

Weaknesses

While this is a strong paper, I see a few areas where the context and implications could be explored further. These are not flaws so much as opportunities for deeper insight.

1. Limited Discussion of the "Fuzzy Matching" Problem: The current approach relies on finding exact repeating sequences of task hashes. The real world of software is often messy. One can easily imagine scenarios with "almost-repeating" patterns, where a main loop body is occasionally interspersed with a conditional logging or debugging task. How robust is Apophenia to such noise? A brief discussion on the limitations of exact matching and potential future work on fuzzy or probabilistic trace identification would place the work in an even broader context.

2. Generality Beyond the Legion Ecosystem: The authors state that their ideas "can be directly applied to other task-based runtime systems" (Section 4, Page 2), but the discussion remains high-level. The implementation is naturally tied to Legion's architecture (e.g., control replication, hashing of region arguments). A more detailed discussion on the core assumptions Apophenia makes about a runtime would be valuable. What specific interfaces or properties must a runtime like StarPU, PARSEC, or Ray expose to support such a "JIT for analysis"? This would strengthen the claim of generality and provide a roadmap for others.

3. The Gap Between the Ideal and the Heuristic: The paper does a good job formally defining the optimization problem for finding good traces in Section 3. The algorithm presented in Section 4.2 is a practical, greedy heuristic to find a good solution efficiently. While perfectly acceptable for a systems paper, it would be interesting to briefly touch on the nature of this trade-off. Is there any theoretical or empirical insight into how far the greedy solution might be from the optimal coverage defined in Section 3?

---

Questions to Address In Rebuttal

1. Could you elaborate on the robustness of the system to "noisy" task streams? For example, if a loop issues a slightly different (but still traceable) sequence of tasks every N iterations due to some infrequent conditional logic, can Apophenia discover the dominant pattern, or does the noise prevent the formation of any long traces?

2. Beyond Legion's specific implementation of tasks and logical regions, what are the fundamental prerequisites for applying Apophenia's approach to another runtime system? Specifically, what properties must the "tasks" and their "arguments" have to be meaningfully tokenized and hashed for your string analysis to be valid?

3. The paper settles on a single set of parameters for the buffer sampling strategy (e.g., using the ruler function) for all experiments. How sensitive is the final performance to these choices? For example, how does the time-to-reach-steady-state (as shown in Figure 9, Page 11) change with a different multi-scale factor or history buffer size? This would provide insight into the robustness of the "one-size-fits-all" approach.

Review Form: The Innovator

---

Summary

The authors present Apophenia, a system designed to automatically identify repeated sequences of tasks (traces) in implicitly parallel, task-based runtime systems. The central idea is to obviate the need for brittle and non-composable manual trace annotations. The core mechanism treats the stream of tasks issued by an application as a string of tokens, generated by hashing each task and its arguments. Apophenia then employs a novel string analysis algorithm to find repeated, non-overlapping substrings in this token stream, which correspond to candidate traces. These candidates are then used to memoize and replay the results of expensive dynamic dependence analysis, thereby reducing runtime overhead.

---

Strengths

The primary strength of this paper lies in its novel formulation of the automatic trace identification problem.

1. Novel Problem Framing: The central thesis—that automatic trace detection can be framed as an online, repeated substring analysis problem on a stream of task hashes—is, to my knowledge, novel. This approach elegantly sidesteps the primary limitation of prior work in related domains (e.g., JIT compilation), which overwhelmingly relies on static code structure like basic blocks, loops, and function entry points. Apophenia operates on a semantically flat stream of high-level operations, which is a fundamentally different and more challenging context.

2. Novel Algorithmic Contribution: The algorithm presented in Section 4.2 and detailed in Algorithm 2 for finding non-overlapping repeated substrings with high coverage appears to be a novel heuristic. While it is built on standard primitives like suffix and LCP arrays, the specific logic for processing adjacent suffix array entries—particularly the method for handling and splitting overlapping repeats (lines 9-14)—is tailored to the specific optimization goal defined in Section 3. It is not a standard textbook algorithm for maximal repeat finding, but a pragmatic O(n log n) solution to their specific coverage problem.

3. Principled Approach to a Practical Problem: The paper successfully translates a practical engineering problem (brittle manual annotations) into a formal optimization problem (Section 3) and then proposes a concrete, efficient algorithmic solution. The application of the ruler function sequence (Section 4.4) to manage the trade-off between trace quality and detection latency is a clever and non-obvious application of a known concept to this new domain.

---

Weaknesses

My critiques are focused on the precision and contextualization of the claimed novelty, rather than its existence.

1. Insufficient Contextualization of the String Algorithm: While Algorithm 2 appears novel in its specifics, its relationship to the broader literature on string covering problems, sequence mining, and motif finding is not fully explored. The paper effectively dismisses tandem repeats and LZ-style compression algorithms, but the problem of finding a set of substrings to achieve maximum disjoint coverage of a larger string is related to established problems in bioinformatics and data compression. A more thorough discussion of how Algorithm 2 relates to, and differs from, existing heuristics for these (often NP-hard) problems would strengthen the claim of algorithmic novelty.

2. Heuristics Lack Formal Justification: The system's effectiveness relies on several heuristics that, while pragmatic, are presented without deep justification. Specifically, the scoring function described in Section 4.3 (length * capped, decayed count + replay bias) is an engineered solution. The paper demonstrates that it works, but does not explore the design space or provide evidence that this specific combination of factors is fundamentally better than simpler alternatives (e.g., a pure length * frequency metric). This element feels more like well-tuned engineering than a fundamental contribution.

3. Abstraction via Hashing: The conversion of a task stream to a token stream via hashing is the foundational abstraction. However, the implications of this abstraction are not discussed. The possibility of hash collisions, while likely rare with a 64-bit hash, is non-zero. A collision could cause the system to incorrectly identify a trace, leading to either a runtime error (if the runtime validates the replayed trace) or silent correctness issues. While not a major flaw, the lack of discussion on the robustness of this core abstraction is a minor weakness.

---

Questions to Address In Rebuttal

1. Can the authors please elaborate on the novelty of Algorithm 2 with respect to the literature on string covering problems? Is this a novel heuristic for a known (potentially NP-hard) problem, or a solution to a newly formulated one? Providing citations to the closest-related formal problems would help situate the contribution.

2. The scoring function for trace selection is critical for balancing exploration and exploitation. Could the authors provide more insight or an ablation study justifying the necessity of its components (decay, capping, replay bias) over a simpler baseline?

3. Regarding the hashing of tasks and their arguments into tokens (Section 4.1): Have the authors analyzed the sensitivity of Apophenia to hash collisions? Could a collision cause the system to attempt to form an invalid trace, and if so, how is this handled by the underlying runtime (e.g., Legion)?

Paper: BatchZK: A Fully Pipelined GPU-Accelerated System for Batch Generation of Zero-Knowledge Proofs Review Form: The Guardian

---

Summary

This paper presents BatchZK, a GPU-accelerated system designed to optimize the throughput of batch zero-knowledge proof (ZKP) generation. The authors correctly identify a gap in existing research, which has largely focused on reducing the latency of single proof generation. Their approach is to construct a deep pipeline where individual computational modules of modern ZKP protocols (sum-check, Merkle tree, linear-time encoder) are broken into stages, with each stage mapped to dedicated GPU resources. The system is evaluated on several GPUs and demonstrated in a verifiable machine learning context.

While the engineering effort is commendable and the raw throughput numbers appear impressive, my assessment is that the paper’s core claims are built on a foundation of questionable and potentially misleading comparisons. The experimental methodology conflates architectural improvements with algorithmic ones, the system's design appears brittle and manually tuned, and the significant negative impact on latency is not adequately addressed.

---

Strengths

1. Relevant Problem Formulation: The paper's focus on throughput for batch ZKP generation is highly relevant for many practical applications, such as large-scale verifiable computation services or blockchain rollups, where aggregating and proving many transactions or computations is the primary goal. This is a valuable shift from the common focus on single-proof latency.
2. Focus on Modern Protocols: The system is designed around the computational primitives of recent, efficient ZKP systems (e.g., those based on the sum-check protocol), rather than the more heavily studied NTT/MSM-based protocols. This aligns the work with the current trajectory of ZKP research.
3. Demonstrated High Throughput: The results, on their face, show a system capable of generating a high volume of proofs per second, particularly in the verifiable machine learning application (Section 6.3, page 13), where sub-second proof generation is achieved.

---

Weaknesses

1. Fundamentally Flawed Performance Comparisons: The paper’s headline claims of massive speedups are predicated on an apples-to-oranges comparison. The abstract claims ">259.5x higher throughput compared to state-of-the-art GPU-accelerated systems." This figure is derived from Table 8 (page 13), which compares BatchZK (implementing modern, cost-effective protocols) against Bellperson (implementing older, more computationally expensive R1CS-based protocols using MSM/NTT). A significant, and likely dominant, portion of this speedup comes from the choice of ZKP protocol, not the authors' proposed pipelined architecture. The authors briefly acknowledge this in their "breakdown" analysis (Section 6.3, page 12), but this admission is buried, while the misleading top-line numbers are highlighted in the abstract and introduction. A rigorous evaluation would compare the proposed pipelined system against a non-pipelined, batched baseline that implements the exact same ZKP protocol. Without this, the true contribution of the pipeline architecture is impossible to quantify.

2. Brittle, Manually-Tuned Resource Allocation: The core of the pipeline's efficiency relies on balancing the stages. The authors state, "we manually allocate resources to different modules to keep their throughput consistent" (Section 4, page 9), providing a hard-coded ratio of 35:12:113 for a V100 GPU. This raises serious concerns about the system's generality and robustness:

   • Architecture Dependence: How does this optimal ratio change for different GPU architectures with varying numbers of SMs, memory bandwidth, and core designs (e.g., A100, H100)? The paper provides no analysis.
   • Problem Size Dependence: How does the ratio change for different problem sizes (e.g., circuit scale S, Merkle tree depth)? A system that requires expert manual re-tuning for every new hardware target or problem class is not a general-purpose solution but a highly specialized proof-of-concept.

3. Insufficient Analysis of the Latency-Throughput Trade-off: The paper's own results in Table 6 (page 11) show that the pipelined approach leads to a substantial increase in per-proof latency compared to non-pipelined baselines (e.g., Merkle tree latency is 2.5x worse than Simon, Sumcheck is 3x worse than Icicle). The authors frame this simply as a trade-off. However, this is a critical limitation that is not explored. Many high-throughput scenarios, such as blockchain transaction sequencing, still have latency constraints. The paper fails to define the application space where such a severe latency penalty is acceptable, weakening the argument for the system's practical impact.

4. Oversimplified Memory Analysis: The claim of reduced memory usage is, again, based on a flawed comparison with Bellperson (Table 10, page 13), whose memory footprint is dominated by its underlying cryptographic operations. The true comparison should be against a naive batching of the same protocols BatchZK uses. While the dynamic loading approach is sensible, the paper provides no analysis of the pipeline's own memory overhead, such as the memory required for buffering intermediate data between the numerous pipeline stages.

5. Missing Experimental Details for Reproducibility: The verifiable machine learning application results (Table 11, page 13) are presented without a key piece of information: the size of the resulting arithmetic circuit (e.g., number of multiplication gates) for the VGG-16 inference. Without this parameter, the reported throughput of 9.52 proofs/second is not comparable to other works and the result is not reproducible.

---

Questions to Address In Rebuttal

1. Regarding the headline performance claims (>259.5x speedup): Can the authors provide a revised evaluation that compares BatchZK against a non-pipelined (but still batched and GPU-accelerated) version of the same sum-check-based protocol? This is the only way to isolate and accurately measure the performance contribution of the proposed pipelining architecture itself.

2. Regarding resource allocation: Please provide a sensitivity analysis for the manual 35:12:113 thread allocation ratio. How does performance degrade when this ratio is applied to different GPUs (e.g., the A100 or H100 from your tests) or to different circuit sizes? Does your system provide any mechanism for automatically determining these ratios?

3. Regarding latency: Please provide a more detailed discussion of the application scenarios for which the observed 2.5-3x degradation in single-proof latency is acceptable. Conversely, are there high-throughput applications (e.g., time-sensitive financial transaction batching) for which BatchZK would be fundamentally unsuitable due to this latency penalty?

4. Regarding the verifiable machine learning evaluation: To ensure reproducibility and fair comparison, please state the exact number of constraints and multiplication gates in the arithmetic circuit generated for the VGG-16 model used in your evaluation in Table 11.

Reviewer: The Synthesizer (Contextual Analyst)

---

Summary

The paper presents BatchZK, a GPU-accelerated system designed for high-throughput batch generation of zero-knowledge proofs (ZKPs). The authors identify a crucial shift in the ZKP landscape: a move away from protocols reliant on expensive primitives like NTT and MSM, towards newer, more efficient protocols built on cost-effective modules like sum-check, Merkle trees, and linear-time encoders. The core contribution is a fully pipelined architecture that decomposes these modern ZKP modules into sequential stages, with each stage mapped to dedicated GPU kernels. This design maximizes GPU utilization for batch processing, directly targeting throughput rather than the single-proof latency optimized by prior works. The authors demonstrate the system's effectiveness with impressive results, including a >259x throughput improvement over prior GPU-accelerated systems and, most notably, achieving sub-second proof generation for a VGG-16 model in a verifiable machine learning application.

---

Strengths

This is an excellent and timely systems paper that makes a significant contribution to the practical application of zero-knowledge proofs.

1. Clear and Insightful Problem Framing: The authors have done a masterful job of contextualizing their work. The distinction they draw between the "first category" (NTT/MSM-based) and "second category" (sum-check/Merkle-based) of ZKP protocols is lucid and compelling (Figure 1, page 3). This framing immediately establishes a clear research gap: hardware acceleration has historically focused on the former, leaving the increasingly popular latter category underserved. This paper squarely addresses that gap.

2. Elegant and Well-Motivated Technical Approach: The core idea of creating a deep pipeline by breaking down ZKP primitives into their constituent computational layers (e.g., layers of a Merkle tree, rounds of a sum-check) is a classic systems optimization applied brilliantly in a new domain. As illustrated in Figure 4 (page 5), this approach directly tackles the GPU underutilization problem inherent in naive parallelization of these "reducing" workloads. It represents a fundamental shift from optimizing for latency to optimizing for throughput, which is the correct metric for many real-world ZKP applications like blockchain rollups and Machine-Learning-as-a-Service (MLaaS).

3. Transformative Performance and Impact: The results are not merely incremental; they are transformative. The ability to generate 9.52 proofs per second for a VGG-16 inference (Table 11, page 13) is a landmark achievement. It moves verifiable machine learning from a theoretical curiosity with multi-minute or multi-second proof times into the realm of practical, sub-second viability for real-time services. This result alone could significantly accelerate the adoption of verifiable AI. The broader system speedups (>259x over Bellperson on a V100 GPU) further underscore the power of their pipelined approach.

4. Bridging Theory and Practice: This work serves as a vital bridge between the fast-moving frontier of ZKP theory and the practical needs of system deployers. By building a high-performance engine for the primitives used in modern protocols like HyperPlonk, Orion, and Virgo, the authors are providing the tooling necessary to unlock the potential of this recent cryptographic research. The paper is an exemplar of how systems and architecture research can act as a force multiplier for advances in other fields.

---

Weaknesses

The weaknesses of the paper are minor and relate more to exploring the boundaries of the proposed approach rather than any fundamental flaws in it.

1. Limited Discussion on Generalizability and Adaptability: The paper demonstrates a highly effective system, but the pipeline appears to be carefully hand-tuned. The resource allocation section (Section 4, page 9) mentions manually balancing threads based on an empirically derived execution time ratio (35:12:113). This raises questions about the engineering effort required to adapt BatchZK to new ZKP protocols or even variations of existing ones. A discussion on the principles of this pipeline construction and the potential for a more automated or adaptable framework would strengthen the paper's contribution from a methodological standpoint.

2. Insufficient Exploration of the Latency-Throughput Trade-off: The authors correctly identify that their system prioritizes throughput over latency and provide some latency numbers in Table 6 (page 11). However, this critical trade-off could be explored more deeply. For many interactive applications, even within a batch-processing paradigm, the "time to first proof" or the end-to-end latency for a given task remains important. A characterization of how latency behaves as batch size and pipeline depth increase would provide a more complete picture of the system's performance envelope and help practitioners better understand its suitability for different use cases.

---

Questions to Address In Rebuttal

1. Regarding the manual resource allocation mentioned on page 9: How sensitive is the system's performance to the precise thread allocation ratios between the different modules (linear-time encoder, Merkle tree, sum-check)? Could the authors elaborate on the process or methodology for tuning these parameters when targeting a new ZKP protocol or a different GPU architecture?

2. The paper's primary contribution is a dramatic improvement in throughput, at the cost of some latency for individual proofs. Could the authors provide more insight into this trade-off? For example, what is the effective end-to-end latency for a proof that enters the pipeline when it is already full (i.e., the steady-state latency)? How does this compare to the latency of a single-proof, non-pipelined execution of the same primitives on the GPU? This would help clarify the exact nature of the performance trade-off.

Paper Title: BatchZK: A Fully Pipelined GPU-Accelerated System for Batch Generation of Zero-Knowledge Proofs Review Form: The Innovator

---

Summary

The authors propose BatchZK, a GPU-accelerated system designed to maximize the throughput of batch zero-knowledge proof (ZKP) generation. The core thesis is that prior GPU work on ZKPs has incorrectly focused on latency reduction for single proofs using older, computationally expensive primitives (NTT, MSM). This paper pivots to address throughput for batches of proofs using modern, cost-effective primitives: the sum-check protocol, Merkle trees, and linear-time encoders. The claimed novelty is the system's "fully pipelined" architecture, where each ZKP primitive is decomposed into a series of dedicated GPU kernels, allowing batches of proofs to be streamed through the system, maximizing hardware utilization.

---

Strengths

The primary strength of this work lies in identifying a gap in the state-of-the-art and proposing a novel system architecture to fill it. While the constituent ideas (pipelining, GPU acceleration) are not new in isolation, their synthesis and application in this specific context represent a genuine contribution.

1. Novel Application of a Classic Architecture: The core idea—decomposing a complex computation into a deep, multi-kernel pipeline for streaming workloads on a GPU—is a novel application in the ZKP domain. Prior GPU-ZKP work like GZKP [38] focused on optimizing monolithic kernels for latency. BatchZK's contribution is to correctly identify that the batch generation problem is a throughput problem, for which a pipeline is the appropriate architectural pattern.

2. Novel Primitive-Specific Decompositions: The novelty is most evident in how the authors adapt each primitive to the pipeline model:

   • Merkle Tree (Section 3.1, page 5): The proposed method of dedicating kernels to specific layers of the tree and streaming multiple trees through this pipeline (Figure 4b) is a clear and novel departure from the "intuitive" single-kernel-per-tree approach used in prior work like Simon [51].
   • Linear-time Encoder (Section 3.3, page 7): The authors state that no prior GPU implementation exists, making their implementation the first. More significantly, their method for handling the inherent recursion by splitting the process into two separate, interconnected pipelines (Figure 6) is a novel and clever transformation that makes the algorithm amenable to a streaming architecture. This is a non-trivial contribution.

3. Clear Distinction from Prior Pipelined ZKP Work: The authors correctly cite PipeZK [64], which applied pipelining to ZKPs on ASICs. However, the delta is significant and clear: PipeZK targeted the older NTT/MSM primitives on a fundamentally different hardware architecture. BatchZK's contribution is a novel pipeline design for the different computational patterns of sum-check, Merkle trees, and linear-time encoders on GPUs. This distinction is well-defined and defended.

---

Weaknesses

The paper's claims of novelty are generally strong, but could be tempered with more precise language regarding which components are truly new versus which are standard high-performance computing (HPC) engineering practices.

1. Overstated Novelty of Standard Optimizations: The paper presents certain techniques as integral parts of its contribution without sufficiently acknowledging them as standard practice. For instance, the use of multi-stream technology to overlap data transfers with computation (Section 4, page 9) is a canonical GPU optimization pattern. Similarly, the double-buffering memory access pattern for the sum-check protocol (Figure 5, page 6) is a well-known technique to avoid memory hazards in streaming algorithms. While their application is necessary for the system to function, these are not novel ideas in themselves. The paper would be stronger if it framed these as "applying standard techniques" rather than implying they are part of the core novel design.

2. Novelty is Architectural, Not Algorithmic: To be clear, this work does not introduce a new ZKP protocol or a new fundamental algorithm for any of the computational modules. Its novelty is purely at the systems and architecture level—it presents a new way to execute existing algorithms. This is a valuable contribution for a systems conference like ASPLOS, but the paper should maintain this clarity throughout. The contribution is in the "how," not the "what."

---

Questions to Address In Rebuttal

1. On the Novelty of the Sum-check Pipeline: The authors contrast their pipelined sum-check module with the implementation in Icicle [28]. The paper's argument rests on the assumption that Icicle uses a monolithic kernel that leads to thread idling. Can the authors provide more concrete evidence for this? A comparative profile of GPU core utilization between their single-proof, non-pipelined sum-check kernel and the Icicle implementation would substantially bolster the claim that their pipelined decomposition is a fundamentally new and superior approach.

2. Generalizability of the Pipeline Transformation: The transformation of the recursive linear-time encoder into a dual-pipeline structure is the most technically novel part of the paper. Is this a bespoke solution tailored exclusively to the Spielman encoder, or is it a generalizable pattern for mapping a certain class of recursive algorithms onto streaming hardware? If the latter, discussing this pattern's characteristics would elevate the significance of this contribution beyond the immediate ZKP context.

3. On the Dynamic Loading Method: The paper claims a "dynamic loading method" as a feature (Abstract, page 1). From the description, this appears to be a direct consequence of the pipelined design (i.e., only data for the current proof-in-flight needs to be loaded). Is there any novel mechanism in the loading method itself, or is the memory reduction simply an inherent benefit of a streaming architecture over a naive parallel approach that would load all m proofs' data at once? Please clarify the precise novelty here.

Paper Title: ByteFS: System Support for (CXL-based) Memory-Semantic Solid-State Drives Reviewer: The Guardian (Adversarial Skeptic)

---

Summary

The authors present ByteFS, a file system co-designed with custom SSD firmware to support memory-semantic SSDs (M-SSDs). The core contribution is a system that utilizes a dual byte/block interface to optimize I/O accesses. Key techniques include: 1) adapting the I/O interface (byte vs. block) based on the data structure and access pattern, 2) modifying the SSD firmware to manage the on-device DRAM as a log-structured write buffer to coalesce byte-granular writes before flushing to NAND flash, and 3) a coordinated caching scheme that prioritizes the host page cache for read caching to reserve SSD DRAM for writes. The authors implement and evaluate ByteFS on both an FPGA-based SSD prototype and an emulator, demonstrating significant throughput improvements (up to 2.7x) and write traffic reduction (up to 5.1x) compared to existing block-based (Ext4, F2FS) and persistent memory file systems (NOVA, PMFS).

Strengths

The paper is technically dense and presents a comprehensive system-level effort, spanning from file system design to firmware modification and hardware prototyping. This level of vertical integration is commendable.

1. Hardware Prototype: The implementation and evaluation on a real, programmable OpenSSD FPGA board (Section 4.9, page 9) lends significant credibility to the performance results, moving beyond pure emulation which can often hide real-world system complexities.
2. Problem Motivation: The quantitative study in Section 3 (pages 3-5) effectively illustrates the well-known problem of I/O amplification in existing file systems, providing a solid, data-driven motivation for the need for a dual-interface approach.
3. Performance Breakdown: The ablation study presented in Figure 12 (page 12) is a crucial piece of analysis. It successfully disentangles the performance contributions of the three main design components (dual interface, log-structured firmware, and adaptive data I/O), which strengthens the authors' claims about the efficacy of each component.

Weaknesses

Despite the strengths, the work rests on several questionable design choices and the evaluation lacks the rigor to fully substantiate its claims. The core assumption that the proposed complexity is a net-win is not convincingly proven.

1. Prohibitive Overhead of Adaptive Data I/O: The mechanism for selecting the data interface, detailed in Section 4.6 (page 8), is a critical flaw. The use of a copy-on-write (CoW) mechanism within the page cache, followed by an XOR comparison to detect modified cache lines, introduces unacceptable overheads. The authors admit that "duplicated pages occupy 16% of the entire page cache size on average," which is a substantial and potentially prohibitive memory overhead, especially in memory-constrained environments. Furthermore, the CPU cost of performing XOR on entire pages, while benchmarked in isolation, is not evaluated as a system-level overhead that consumes cycles that could be used by the application. The marginal gains shown for this mechanism in Figure 12 for workloads like OLTP do not appear to justify this complexity and cost.
2. Insufficient Evaluation of Background Work: The log-structured SSD DRAM is central to the design, yet the impact of its background log cleaning process (Section 4.3, page 7 and Algorithm 1) is not adequately evaluated. The authors admit that cleaning can involve read-modify-write patterns, leading to higher flash traffic than baselines in some cases (Section 5.3, page 12). They dismiss this by stating it occurs "in the background," but background work is not free; it consumes internal device bandwidth and controller resources, which can create I/O interference and introduce significant tail latency. The evaluation only presents p95 latency (Figure 7, page 11), which is insufficient to expose the impact of such garbage collection-like activities. A rigorous evaluation must include p99 and p999 latencies to demonstrate that log cleaning does not introduce performance cliffs.
3. Ambiguous Persistence Guarantees and Overhead: The mechanism for ensuring the persistence of byte-granular writes relies on a clflush/clwb followed by a "write-verify read" (a zero-byte read) to flush PCIe transaction buffers (Section 4.2, page 6). This is a known technique, but it effectively serializes dependent operations at the PCIe root complex. The cost of this serialization is not measured. For workloads with high-concurrency, small synchronous writes (common in databases and journals), this serialization could become a major bottleneck, undermining the benefits of the byte interface. The paper lacks a microbenchmark that isolates and quantifies this persistence overhead under concurrent load.
4. Conflated Contributions in Baseline Comparison: The primary performance results (Figure 6, page 10) compare ByteFS running with its custom, log-structured firmware against baseline file systems running on the M-SSD with a standard caching firmware. This is not a fair, apples-to-apples comparison of the file systems themselves. It conflates the benefits of the FS design with the benefits of a superior firmware design. The performance breakdown in Figure 12 helps, but the headline claims are based on a comparison that is fundamentally skewed. The baselines are not given the opportunity to run on firmware that is optimized for this class of device.

Questions to Address In Rebuttal

1. Regarding the CoW/XOR mechanism for adaptive data I/O: Can you provide a detailed analysis of the trade-offs? Specifically, what is the measured CPU overhead (as a percentage of total CPU time) and memory overhead under each of the evaluated macro-benchmarks? At what modified page ratio R does the overhead of this mechanism outweigh the benefit of using byte-granular writes?
2. The paper's evaluation of the log-structured DRAM is incomplete. Please provide tail latency data at the 99th and 99.9th percentiles for the YCSB workloads to demonstrate that the background log cleaning process does not introduce unacceptable latency spikes. Furthermore, can you measure the internal SSD bandwidth consumed by the cleaning process and show its impact on foreground I/O performance?
3. Please provide microbenchmark results that specifically measure the throughput and latency of small (e.g., 64-byte) synchronous persistent writes using your clflush/read-verify mechanism. The benchmark should vary the number of concurrent threads to demonstrate how the serialization point at the root complex affects scalability.
4. To create a fairer comparison, could you implement a simplified version of the log-structured write buffer in the firmware and run a traditional file system like Ext4 on top of it? This would help to more clearly isolate the performance gains attributable solely to the ByteFS file system's dual-interface management versus the gains from the superior firmware design.

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper introduces ByteFS, a novel file system co-designed with SSD firmware for emerging memory-semantic solid-state drives (M-SSDs). These devices, often enabled by interconnects like CXL, offer a dual-mode interface: fast, byte-granular memory-mapped access and traditional, high-throughput block-based access. The core contribution of this work lies in its holistic approach to embracing this duality, rather than forcing the device into a purely memory-like or purely block-like model.

ByteFS intelligently partitions filesystem operations, using the byte-addressable interface for small, latency-sensitive metadata updates (e.g., inode fields, bitmap flips) and the block interface for bulk data transfers. To bridge the fundamental mismatch between the byte-accessible host interface and the page-granular nature of internal NAND flash, the authors propose a crucial firmware modification: managing the SSD's internal DRAM as a log-structured write buffer. This allows small byte-writes to be coalesced efficiently before being written to flash, significantly reducing I/O amplification. The system is evaluated on both an FPGA prototype and an emulator, demonstrating substantial performance gains and reductions in write traffic compared to state-of-the-art file systems designed for either block devices or persistent memory.

Strengths

1. Timeliness and Strategic Relevance: The paper addresses a critical and timely problem. With the discontinuation of Intel Optane, the industry is actively seeking practical alternatives for storage-class memory. CXL-attached, memory-semantic SSDs are a leading candidate. This work provides one of the first comprehensive system software blueprints for this new class of hardware, moving the conversation from "can we build it?" to "how should we use it?" It's a forward-looking paper that is well-positioned to influence future system design.

2. Excellent Problem Diagnosis: The quantitative study in Section 3 (p. 3-4) is a standout feature. By meticulously dissecting the I/O patterns of individual filesystem data structures (Table 3), the authors provide a compelling, data-driven justification for their dual-interface design. This analysis clearly shows that a one-size-fits-all approach (either pure byte or pure block) is suboptimal and lays a strong foundation for the design of ByteFS.

3. Pragmatic Hardware/Software Co-Design: The paper's strength is its recognition that the problem cannot be solved in the host software alone. The proposed firmware modifications—specifically, the log-structured DRAM cache (Section 4.3, p. 6)—are the linchpin of the entire system. This co-design elegantly resolves the impedance mismatch between the host's view of the device and the physical reality of its NAND media. It provides a practical path forward that acknowledges the constraints of both hardware and software.

4. Connecting Disparate Concepts: The design of ByteFS is a masterful synthesis of ideas from different domains. It borrows the fine-grained access patterns from persistent memory file systems (like NOVA), the robustness of traditional block-based systems (like Ext4), and the write-efficiency of log-structured systems (like F2FS), but reapplies these concepts in a new context. The decision to implement logging within the device firmware is particularly insightful, as it hides flash-related overheads from the host and simplifies crash consistency logic.

Weaknesses

While the core ideas are strong, the paper could be strengthened by a deeper exploration of the following aspects:

1. Exploration of Design-Space Trade-offs: The paper presents a set of well-motivated heuristics for interface selection (e.g., the 512B threshold for direct I/O, the CoW mechanism for buffered I/O in Section 4.6, p. 8). While the evaluation shows these work well, the paper would benefit from a discussion of the sensitivity to these choices. How does performance change as these thresholds are varied? This would provide a richer understanding of the design space and offer guidance for tuning on different hardware.

2. Scalability of the Recovery Mechanism: The crash recovery process (Section 4.7, p. 9) relies on scanning the in-device transaction log. The paper reports a fast recovery time of 4.2 seconds (Section 5.5, p. 12). However, as CXL devices evolve to include multi-gigabyte DRAM caches, this linear scan could become a bottleneck. A brief discussion of how the recovery mechanism could be scaled—perhaps through checkpointing or more structured indexing within the log—would add to the work's long-term relevance.

3. Positioning Relative to CXL.mem Coherency: The authors make a design choice to use a custom persistence protocol (clflush + write-verify read) rather than relying on the full CXL.mem cache coherency protocol. This is a reasonable choice for compatibility and simplicity. However, the paper misses an opportunity to discuss this trade-off more explicitly. A deeper analysis of the performance, complexity, and hardware-dependency implications of their approach versus a fully coherent one would provide valuable context for other researchers building on this work.

Questions to Address In Rebuttal

1. Regarding the adaptive policies for interface selection (e.g., the 512B threshold in Section 4.6, p. 8): Could the authors elaborate on the sensitivity of the system's performance to this threshold? Is there a case for a more dynamic or workload-aware policy beyond the static threshold and CoW-based ratio?

2. The recovery process described in Section 4.7 (p. 9) involves scanning the log region. While the measured recovery time is short on the prototype, could the authors comment on how this approach scales with a much larger in-device DRAM and log region, as might be common in future CXL devices?

3. The paper chooses a custom persistence mechanism (clflush + write-verify read). Given the CXL context, could the authors provide more rationale for this choice over leveraging CXL.mem's native coherency protocols? What are the key performance or implementation trade-offs that motivated this design decision?

Reviewer: The Innovator (Novelty Specialist)

Summary

This paper presents ByteFS, a novel file system designed for memory-semantic SSDs (M-SSDs) that feature dual byte-addressable (via MMIO) and block-addressable (via NVMe) interfaces. The core contribution is a software/hardware co-design that spans the file system and the SSD firmware. The authors claim novelty in three main areas: (1) an adaptive policy within the file system to dynamically select the appropriate interface (byte or block) based on the access pattern, (2) a firmware-level, log-structured management of the SSD's internal DRAM to efficiently coalesce byte-granular writes into page-granular flash writes, and (3) a coordinated caching scheme that dedicates SSD DRAM to this write log while offloading read caching to the host page cache. The evaluation, performed on a real FPGA-based prototype and an emulator, shows significant performance gains and I/O reduction compared to both traditional block-based file systems and existing persistent memory file systems.

Strengths

The primary strength of this work lies in its holistic, co-designed approach to a compelling new hardware target. The paper correctly identifies that neither existing block-based file systems nor persistent memory file systems are a natural fit for M-SSDs. The novelty of the proposed solution is significant:

1. Novel Co-design for Granularity Mismatch: The central novel idea is the tight coupling between the host file system and the device firmware to resolve the byte-host vs. page-flash access granularity mismatch. While SSDs internally buffer writes, ByteFS makes this buffer an explicit, transactionally-consistent log that is directly coordinated with the host file system via custom commands (e.g., COMMIT(TxID) as discussed in Section 4.3, page 7). This elevates a standard FTL optimization into a first-class primitive for system software.

2. Novel Heuristic for Interface Selection: The mechanism for choosing the access granularity for dirty pages in the buffered I/O path is particularly novel. Using Copy-on-Write (CoW) to track changes and XORing the original and modified pages to quantify the "dirtiness" (Section 4.6, page 8) is a clever, concrete heuristic. This provides a data-driven policy for when to expend byte-granular MMIO writes versus a more efficient block-granular NVMe write, a problem unique to this class of device.

3. Novel Caching Policy: The coordinated caching policy is simple but conceptually novel in this context. The decision to forgo read caching in the SSD DRAM and dedicate that precious resource entirely to a persistent write log (Section 4.3, page 6) is a strong design choice that directly addresses the performance characteristics of the underlying flash media (writes are slow and benefit from coalescing). It avoids the redundancy of caching the same data blocks in both the host page cache and the device DRAM, a clear win.

Weaknesses

My analysis focuses exclusively on novelty. While the overall system is a novel composition of ideas, some of the constituent concepts have appeared in recent literature, which slightly tempers the novelty of the individual components, though not the system as a whole.

1. Overlapping Concept of Firmware-Level Logging: The core idea of using a log-structured buffer in the SSD's DRAM to handle the granularity mismatch for CXL-attached SSDs has been explored in prior work. Specifically, "Overcoming the memory wall with CXL-Enabled SSDs" (Yang et al., USENIX ATC '23) [49] also proposes a firmware-level write log to coalesce writes and hide flash latency. While ByteFS's contribution is the full-fledged POSIX file system built on top of this idea, the foundational firmware concept is not entirely de novo. The paper would be strengthened by explicitly positioning its contribution as the system software integration of this emerging device architecture, differentiating it more clearly from device-level proposals like Yang et al.

2. Incremental Novelty on Dual-Interface Hardware: The concept of a dual-interface byte/block SSD was previously introduced by "2B-SSD" (Bae et al., ISCA '18) [12]. ByteFS is a significant and necessary step forward by providing the file system logic to actually exploit such a device. However, the claim of novelty should be carefully scoped to the software system and co-design, rather than the underlying hardware concept itself. The paper does cite this work, but the delta should be framed as enabling a general-purpose file system, which is a substantial but specific type of advancement over the prior art.

Questions to Address In Rebuttal

1. The work by Yang et al. (USENIX ATC '23) [49] also proposes a firmware-level log in SSD DRAM for CXL SSDs to bridge the granularity gap. Could the authors please clarify the novelty of their firmware design in light of this prior work? Is the novelty primarily in the host-side file system's ability to leverage such a feature, or are there fundamental differences in the firmware log design itself (e.g., the indexing structure, transaction management)?

2. The CoW and XOR mechanism for selecting the writeback interface for dirty pages is an interesting heuristic. What is the novelty of this specific technique? Have similar bitwise comparison techniques been used in other contexts (e.g., data deduplication, differential backup) to guide policy decisions in a file or storage system, and if so, how does your application of it represent a novel contribution?

3. The coordinated caching policy is a key part of the design. Can you elaborate on whether this is a fundamentally new idea, or an application of known cache-coordination principles to the specific, novel context of M-SSDs? The novelty seems to stem from the context; please confirm if that is the correct interpretation.

Reviewer Persona: The Guardian (Adversarial Skeptic)

---

Summary

The paper presents Cinnamon, a cross-stack framework designed to accelerate large-scale machine learning workloads under Fully Homomorphic Encryption (FHE). The authors propose a scale-out, multi-chip architecture as an alternative to the prevailing trend of large monolithic FHE accelerators. The core contributions are: (1) novel parallel keyswitching algorithms ("Input Broadcast" and "Output Aggregation") intended to reduce inter-chip communication, (2) a compiler infrastructure with a DSL to manage program- and limb-level parallelism, and (3) a space-optimized hardware Base Conversion Unit (BCU). The authors claim significant performance improvements, most notably a 36,600x speedup for BERT inference over a CPU baseline and a 2.3x improvement over prior art for smaller benchmarks.

While the problem addressed is of significant importance and the proposed approach is comprehensive, the work rests on several claims that are either insufficiently substantiated or compared against potentially weak baselines. The experimental evaluation, particularly concerning comparisons to prior art and CPU performance, lacks the rigor necessary to fully validate the claimed contributions.

Strengths

1. Problem Significance: The paper correctly identifies a critical bottleneck in the field: the inability of monolithic FHE accelerator designs to keep pace with the exponential growth of ML models (as illustrated in Figure 1). The focus on enabling large models like BERT is timely and ambitious.
2. Comprehensive Approach: The work spans the full stack from algorithms and a compiler to the microarchitecture. This holistic view is commendable, as optimizations at a single level are often insufficient in the FHE domain.
3. Focus on Communication: The central thesis—that reducing communication overhead in limb-level parallel keyswitching is key to a viable scale-out architecture—is fundamentally sound. The identification of keyswitching as the primary obstacle to distributed FHE computation is correct.

Weaknesses

1. Vague and Potentially Unfair CPU Baseline: The headline claim of a 36,600x speedup for BERT (Table 2, page 12) is predicated on a comparison to a "48-core Intel Xeon with a 256GB Memory CPU." This description is critically insufficient. The specific CPU model, clock frequency, and—most importantly—the FHE library (e.g., SEAL, HEAAN, Lattigo) and its version are not specified. Furthermore, it is not stated whether the CPU implementation was optimized to leverage all 48 cores for parallel FHE operations. State-of-the-art FHE libraries have seen significant performance improvements. Without these details, the reported speedup is impossible to verify and may be substantially inflated due to comparison against a non-optimized or outdated baseline.
2. Questionable Characterization of Prior Art (CiFHER): The paper consistently positions its keyswitching algorithms as superior to the approach in CiFHER [38]. The central argument is that CiFHER requires broadcasts at both the mod up and mod down steps, incurring high communication overhead. In Section 7.4 (page 13), the authors claim their method reduces inter-chip communication by 2.25x over CiFHER. However, the analysis lacks depth. It is not clear if the CiFHER implementation used for comparison represents the most optimized version of its broadcast-based reduction scheme. Figure 13 (page 12) shows CiFHER resulting in a slowdown over a sequential single-chip implementation, which is a very strong and somewhat counterintuitive claim that suggests the baseline comparison may not be entirely fair. The algorithmic analysis in Section 7.4 feels simplistic and may mischaracterize the trade-offs made in the CiFHER design.
3. Highly Optimistic Cost Model: The performance-per-dollar analysis in Section 7.2 (page 11) and Table 3 (page 12) relies on a manufacturing yield model with a defect density of Do = 0.2cm⁻². This is an extremely optimistic parameter, especially for a large, complex ASIC on a 22nm process, which is mature but not leading-edge for such designs. This choice of parameter significantly favors the smaller-chip approach of Cinnamon over the larger monolithic designs (Cinnamon-M, CraterLake), potentially exaggerating the cost benefits. The conclusions drawn from this analysis are therefore fragile and sensitive to this key assumption.
4. Impact of the "Space-Optimized" BCU is Not Quantified: The paper introduces a novel BCU design in Section 4.7 (page 9), claiming it reduces area by being proportional to the number of input limbs rather than output limbs. While the design rationale is plausible, its actual impact is never isolated or quantified. Table 1 (page 10) shows the area breakdown, but there is no ablation study or comparison showing how much performance-per-dollar or total cost is actually improved by this specific unit versus, for example, using a CraterLake-style BCU within the Cinnamon scale-out framework. As such, the BCU feels like a secondary contribution whose significance to the overall system-level claims is unproven.
5. Unexamined Scalability Limits of the "Input Broadcast" Algorithm: The "Input Broadcast Keyswitching" algorithm (Figure 8b, page 7) requires broadcasting the entire input polynomial Co to all chips. A ciphertext polynomial Co is a large data structure. The paper evaluates systems with up to 12 chips. The cost of this full broadcast seems tenable for this scale, but the paper fails to analyze its asymptotic complexity or practical limitations as the number of chips scales to 16, 32, or beyond. It is plausible that this broadcast itself becomes the new system bottleneck at a larger scale, limiting the very "scale-out" potential the framework claims to provide.

Questions to Address In Rebuttal

1. Regarding the CPU Baseline: Please provide the exact model of the Intel Xeon CPU, its clock speed, the FHE library and version used for the BERT benchmark, and confirm whether the FHE computation was explicitly parallelized across all 48 cores.
2. Regarding the CiFHER Comparison: Please clarify the specific implementation of the CiFHER parallel keyswitching algorithm used for comparison in Figure 13. Can you provide evidence that this implementation is a fair and optimized representation of the approach described in the original CiFHER paper [38]? Why does it result in a net slowdown?
3. Regarding the Cost Model: Can you justify the choice of Do = 0.2cm⁻² for the yield model? Please provide a sensitivity analysis showing how the performance-per-dollar results in Figure 12 change with more conservative (i.e., higher) defect density parameters, for instance, Do = 0.4cm⁻² or 0.6cm⁻².
4. Regarding the BCU Contribution: Please quantify the end-to-end impact of your space-optimized BCU. How would the overall chip area, cost, and performance-per-dollar of a Cinnamon-4 system change if it were to use a scaled-down version of the BCU from CraterLake [56] instead of your proposed design?
5. Regarding Algorithmic Scalability: Please provide an analysis of the communication complexity of the "Input Broadcast Keyswitching" algorithm as a function of the number of chips (n). At what value of n do you project the initial broadcast of Co to become a limiting factor for performance?

Paper: Cinnamon: A Framework for Scale-Out Encrypted AI Reviewer: The Synthesizer (Contextual Analyst)

---

Summary

This paper presents Cinnamon, a full-stack framework designed to accelerate large-scale machine learning workloads under Fully Homomorphic Encryption (FHE). The authors identify a critical divergence: while ML models are growing exponentially in size and complexity, FHE hardware accelerators have pursued a "scale-up" strategy, resulting in massive, monolithic chips that are already failing to keep pace.

The core contribution of Cinnamon is to reject this monolithic paradigm in favor of a "scale-out" approach. This is not merely a hardware proposal but a holistic co-design spanning a new Python DSL, compiler infrastructure with novel intermediate representations (IRs), innovative parallel algorithms for communication-intensive FHE primitives (notably keyswitching), and a scalable multi-chip hardware architecture. By tackling parallelism at every level of the stack, Cinnamon demonstrates, for the first time, the feasibility of running a large language model like BERT under FHE with practical inference times, a feat unattainable with prior state-of-the-art designs. The work effectively argues that the future of practical FHE acceleration lies in distributed, composable systems rather than in building ever-larger single chips.

Strengths

1. A Necessary and Timely Paradigm Shift: The paper's most significant strength is its central thesis. The authors correctly diagnose that the monolithic, scale-up approach in FHE acceleration is on an unsustainable trajectory, a point powerfully illustrated by their Figure 1 (page 2). By drawing a parallel to the broader history of computing—from single-core frequency scaling to multi-core processors, and from single large machines to distributed HPC clusters—Cinnamon positions itself as a crucial and timely architectural intervention. It provides a compelling vision for how the field can escape the design-cost-yield trap of monolithic chips.

2. Holistic, Cross-Stack Co-Design: This is not just a paper about a faster interconnect or a clever hardware unit. The true innovation lies in the tight integration of solutions across the entire stack. The novel parallel keyswitching algorithms ("Input Broadcast" and "Output Aggregation," Section 4.3.1, page 7) are the theoretical key. The compiler's "Keyswitch Pass" is what makes these algorithms practical by automatically identifying patterns and batching communication. The scale-out hardware is then purposefully designed to provide the specific communication primitives (broadcast, aggregate) that the algorithms require. This synergy is what overcomes the communication bottleneck that limited previous multi-chip/chiplet attempts like CiFHER, and it is a masterclass in system co-design.

3. Demonstration of a Breakthrough Capability: While the speedups on smaller benchmarks are impressive (e.g., 2.3x over SOTA), the qualitative result of running BERT is the paper's crown jewel. By reducing a 17-hour CPU computation to a 1.67-second inference on Cinnamon-12, the authors have fundamentally shifted the goalposts for what is considered a "tractable" FHE workload. This moves privacy-preserving ML for large models from a distant theoretical possibility to a tangible engineering problem. This result alone has the potential to energize the field and attract new research and commercial interest.

4. Pragmatic Economic and Architectural Arguments: The authors supplement their performance claims with a solid analysis of manufacturing costs and yields (Section 7.2, Table 3, page 12). The argument that a system of smaller, higher-yield chips is more economically viable than one massive, low-yield chip is critically important for translating academic research into real-world technology. Furthermore, the introduction of a space-optimized Base Conversion Unit (BCU) in Section 4.7 (page 9) shows a thoughtful, bottom-up approach to reducing the area and power of each individual chip in the scale-out system, reinforcing the overall design philosophy.

Weaknesses

While this is a strong and impactful paper, its positioning within the literature and the exploration of its limitations could be strengthened.

1. Absence of a Direct SOTA Comparison on BERT: The paper's most compelling result—accelerating BERT—is evaluated only across Cinnamon configurations. While the authors rightly imply that prior monolithic accelerators cannot run BERT due to memory limitations, this could be made more explicit and quantitative. A projection of the required on-chip cache and estimated die area for a monolithic design (e.g., CraterLake or ARK) to handle BERT would provide a powerful, even if theoretical, baseline that would further underscore the necessity of the scale-out approach.

2. Generalizability Beyond Embarrassingly Parallel Workloads: BERT, with its attention and feed-forward layers, contains significant opportunities for data parallelism, which the Cinnamon framework exploits beautifully. However, the paper does not discuss how the framework would perform on FHE workloads with more complex, serial dependency graphs where program-level parallelism is scarce. The reliance on user-provided parallel streams via the DSL suggests that performance is heavily tied to the application's structure. A discussion of performance on a less parallelizable FHE algorithm would help to better define the framework's application scope.

3. Scalability of the Network Topology: The evaluation explores systems of up to 12 chips using ring and switch topologies. For a true scale-out vision, it is important to consider the next order of magnitude. How do the communication costs of the proposed keyswitching algorithms scale as the system grows to 32, 64, or more chips? At some point, the all-to-all nature of aggregation and broadcast on simple networks can become a bottleneck. A brief analysis of the network scalability would strengthen the long-term vision of the work.

Questions to Address In Rebuttal

1. To strengthen the headline BERT result, could the authors provide a more direct, even if theoretical, comparison against a scaled-up monolithic architecture? For instance, what would be the projected on-chip memory requirements and die size for a "Cinnamon-M" style chip to handle BERT, and how would this impact its manufacturing feasibility and cost according to your model in Section 7.2?

2. The BERT benchmark showcases significant data parallelism. Could the authors comment on how Cinnamon's performance and parallelization strategies would apply to FHE applications with more intricate, sequential dependency graphs where program-level parallelism is less abundant? Does the framework's effectiveness hinge on the availability of such parallelism?

3. Your evaluation focuses on up to 12 chips. Have you analyzed the potential communication bottlenecks in the proposed ring or switch topologies when scaling to significantly larger systems (e.g., 32+ chips)? How do the specific broadcast/aggregate communication patterns of your parallel FHE algorithms scale with these network designs?

Reviewer Persona: The Innovator

Summary

This paper presents Cinnamon, a co-designed framework consisting of new algorithms, a compiler, and a scale-out hardware architecture for accelerating large-scale machine learning workloads under Fully Homomorphic Encryption (FHE). The central thesis is that the traditional monolithic "scale-up" approach for FHE acceleration is not sustainable. Instead, the authors propose a "scale-out" approach using multiple smaller, more cost-effective chips.

The core of the claimed novelty lies in two areas: 1. Algorithmic/Compiler: A set of new parallel keyswitching algorithms ("Input Broadcast" and "Output Aggregation") and compiler passes designed to minimize the inter-chip communication that has historically been the primary obstacle to efficient limb-level parallelism. 2. Architectural: A space-optimized Base Conversion Unit (BCU) that reduces chip area by exploiting specific characteristics of FHE workloads.

The framework is used to demonstrate, for the first time, a practical inference time for a BERT-sized model, which serves as evidence for the efficacy of the proposed scale-out methodology.

Strengths

The paper's primary strength lies in its identification and proposed solution for the communication bottleneck in limb-level parallel FHE. While the concepts of program-level and limb-level parallelism are not new in themselves, the specific techniques developed here represent a genuine advancement over the prior art.

1. Novel Parallel Keyswitching Algorithms: The most significant contribution is the design of the "Input Broadcast" and "Output Aggregation" keyswitching algorithms (Section 4.3.1, page 7). Prior work in multi-chip FHE, notably CiFHER [38], relied on extensive broadcasting of data, which does not scale well with increasing communication latency or bandwidth constraints. Cinnamon's approach of strategically choosing a single communication point (either at the beginning or end of the operation) and then using compiler transformations to batch these communication events across many operations is a fundamentally new and more scalable method. The algorithmic analysis in Section 7.4, which argues for a reduction in communication complexity from O(r) to O(1) for batched rotations, clearly articulates this novel delta.

2. Novel Domain-Specific Microarchitecture: The design of the space-optimized Base Conversion Unit (BCU) is a clever and novel architectural contribution (Section 4.7, page 9). Prior designs like CraterLake [56] apparently followed a general-purpose, output-buffered approach. The authors correctly identify that FHE base conversions are asymmetric (few input limbs to many output limbs) and exploit this by designing an input-buffered unit. This insight leads to a direct and substantial reduction in the required logic and SRAM resources, making the individual chips in the scale-out system more area- and cost-efficient. This is a prime example of a valuable, domain-specific hardware optimization.

3. Co-design of Compiler and Algorithms: The paper presents a well-realized co-design. The parallel algorithms were clearly designed with the awareness that a compiler could reorder and batch operations, and the "Cinnamon Keyswitch Pass" (Section 4.3.1) is the embodiment of this synergy. This tight integration is what elevates the work from a collection of point optimizations to a cohesive and novel framework.

Weaknesses

From a novelty perspective, the weaknesses are primarily in areas where the contributions are more evolutionary than revolutionary, or where the framing could be sharpened to better distinguish from existing concepts.

1. Incremental Nature of Program-Level Parallelism Abstractions: The use of a Python DSL and concurrent execution streams (Section 4.2) to express program-level parallelism is a standard practice in the broader parallel computing domain. While its application to FHE is necessary for the framework, the abstraction itself is not fundamentally new. The novelty is less in the DSL and more in how the compiler backend maps these streams onto the scale-out hardware using the novel limb-level parallel techniques.

2. Scale-Out Concept Follows Prior Art: The paper is correctly motivated by the need to move from "scale-up" monolithic chips (e.g., CraterLake) to "scale-out" multi-chip systems. However, the first step in this direction within FHE hardware was taken by CiFHER [38], which introduced a chiplet-based design. Therefore, Cinnamon's core idea is not to scale out, but rather how to scale out efficiently. The paper is mostly clear on this, but the high-level framing should consistently emphasize that the novelty is in the enabling mechanisms (the new algorithms) that make scaling out practical, rather than the idea of scaling out itself.

3. Demonstration of BERT is an Application, Not a Core Novelty: The paper rightly highlights the impressive achievement of running BERT inference in 1.67 seconds. However, this is an experimental result that validates the framework's novelty; it is not, in itself, a novel conceptual contribution. This result stems directly from the novel algorithms and architecture and should be presented as such, rather than as a standalone claim of novelty.

Questions to Address In Rebuttal

1. The proposed keyswitching algorithms trade communication for some duplicated computation and storage of extension limbs. Could the authors provide a more formal analysis of this trade-off? Specifically, is there a crossover point in terms of the number of chips, network bandwidth, or FHE parameters where the overhead of this duplication would outweigh the communication benefits, potentially favoring a CiFHER-like broadcast approach?

2. The "Input Broadcast" keyswitching algorithm (Figure 8b) appears to require each chip to store a full copy of the input polynomial CQ after the broadcast. Given that ciphertexts can be large, does this place a significant new memory capacity requirement on each chip compared to prior approaches, and how does this scale with the number of limbs?

3. The concept of reordering and batching communication is related to hoisting techniques described in the context of software libraries like HElib [28]. Can you more precisely delineate the novelty of your compiler's "Keyswitch Pass" from the principles used in prior software-based FHE optimization, especially in how it handles the explicit costs of a distributed multi-chip system?

Paper Title: CLOSUREX: Compiler Support for Correct Persistent Fuzzing Reviewer: The Guardian (Adversarial Skeptic)

---

Summary

The authors present CLOSUREX, a compiler-based instrumentation framework designed to enable high-performance, semantically correct persistent fuzzing. The core idea is to eliminate the process management overhead (e.g., fork) inherent in traditional, correct fuzzing approaches. CLOSUREX achieves this by instrumenting the target program at the LLVM IR level to track and reset program state between test case executions within a single, long-running process. The state restoration targets global variables, the program stack, heap allocations, and file descriptors. The evaluation, conducted on ten benchmarks, claims that CLOSUREX achieves a 3.5x speedup in executions per second over AFL++'s forkserver mode, finds bugs 1.9x faster, and maintains semantic correctness equivalent to that of a fresh process execution.

Strengths

1. Problem Significance: The paper addresses a well-understood and significant bottleneck in fuzzing: the performance cost of process creation and initialization. The motivation to bridge the performance gap between incorrect persistent fuzzing and correct fork-based fuzzing is strong.

2. Sound Core Concept: The approach of using compile-time instrumentation to manage state rollback is a logical and powerful technique. It correctly identifies the primary sources of state pollution in many C-based programs.

3. Impressive Performance Results: A 3.5x average increase in test case throughput (Table 5, p. 9) over the state-of-the-art forkserver is a substantial performance gain. If correct, this is a significant engineering achievement.

4. Tangible Bug-Finding Impact: The discovery of 15 0-day bugs, including 4 CVEs (Abstract, p. 1), provides strong evidence that the tool is practically effective, at least on the selected targets.

Weaknesses

My primary concerns with this submission revolve around the strength and generalizability of its correctness claims, the limitations of its state restoration model, and the robustness of its bug-finding evaluation.

1. Overstated and Unsubstantiated Correctness Claims: The central premise of the paper is "correctness." However, the validation of this claim in Section 6.5 (p. 10) is methodologically weak and does not support the strong assertions made.

   • The authors' method for verifying equivalence relies on running queued test cases and comparing dataflow (heap state) and control-flow (path coverage) against a fresh process execution.
   • Critically, the authors state they "exclude test cases that induce a non-deterministic execution path on the target." This is a fatal flaw in a proof of general correctness. Fuzzing inherently explores complex, often-unstable program behaviors, including those involving uninitialized data, certain PRNGs, or environmental interactions, which can be non-deterministic. By excluding these cases, the evaluation demonstrates correctness only for the subset of well-behaved, deterministic executions, fundamentally undermining the claim of general semantic equivalence. The claim of "maintaining semantic correctness" (Abstract, p. 1) is therefore not fully supported.

2. Incomplete State Restoration Model: The paper claims to provide "whole-application persistent fuzzing" (Section 4, p. 5), but the described state restoration is partial. CLOSUREX handles globals, stack, malloc/free, and fopen/fclose. This neglects numerous other critical sources of process state that can cause cross-test-case contamination:

   • Memory Maps: State created via mmap is not discussed. A target that maps a file, modifies it in-memory, and does not munmap it will pollute subsequent test cases.
   • IPC and Networking: Sockets, shared memory segments, pipes, and other forms of inter-process communication are not handled. A server application under test would almost certainly enter an invalid state.
   • Static State in Libraries: The function replacement technique for malloc and fopen will fail if a statically-linked, pre-compiled library makes direct syscalls or uses its own internal state/allocators that are not visible at the LLVM IR level of the main application. The authors acknowledge this as future work in Section 7.4 (p. 11), but it is a fundamental limitation of the current approach and its claims. The "correctness" is conditional on a very specific and limited programming model.

3. Weak Bug-Finding Metric: The claim that CLOSUREX "finds bugs more consistently and 1.9x faster than AFL++" (Abstract, p. 1) is based on the "time-to-first-bug" metric (Table 7, p. 10). This metric is notoriously noisy and can be misleading. A superior evaluation would compare the total number of unique crashes or unique code paths discovered by each fuzzer over the entire 24-hour campaign. It is possible that while CLOSUREX finds the first bug faster due to raw speed, AFL++'s different execution pacing might explore a different, ultimately more productive part of the state space over the long term. Without this data, the claim of superior bug-finding effectiveness is weak.

4. Ambiguous Comparison Baseline: The paper rightly positions AFL++'s forkserver as the primary correct baseline. However, it dismisses AFL++'s own persistent mode as incorrect without providing a performance comparison. While the mode is indeed fragile, quantifying the performance gap between it and CLOSUREX would clearly demonstrate how much of the "unsafe speed" has been recovered "safely." Similarly, kernel-based snapshotting is dismissed on portability grounds (Table 2, p. 4), but its performance on a supported configuration is not compared. Is CLOSUREX faster than all correct approaches on their optimal platforms? The paper does not provide the evidence to support such a strong claim.

Questions to Address In Rebuttal

1. Given that your correctness evaluation explicitly excludes non-deterministic test cases, how can you justify the general claim that CLOSUREX "maintains semantic correctness"? Please re-frame the contribution to accurately reflect that correctness has only been demonstrated for deterministic program paths.

2. Please provide a more exhaustive list of stateful behaviors your system does not currently handle (e.g., mmap, sockets, direct syscalls from linked libraries, dlopen). How would the presence of any of these in a target program break CLOSUREX's correctness guarantees?

3. To substantiate the claim of superior bug-finding, please provide data comparing the total number of unique crashes (or a similar metric like unique edges/paths) discovered by CLOSUREX and AFL++ over the full 24-hour fuzzing campaigns, rather than just the time to the first crash. A time-series plot would be most effective.

4. Can you clarify the practical limitations of your approach regarding the fuzzing of complex, real-world applications that rely heavily on pre-compiled third-party libraries (e.g., OpenSSL, zlib)? If all source code must be available and instrumented, this should be stated as a primary constraint of the system.

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper presents CLOSUREX, a compiler-based system designed to solve a fundamental trade-off in fuzzing: the choice between the high performance of persistent fuzzing and the semantic correctness of fresh-process execution. The authors correctly identify that while persistent mode offers the highest throughput by reusing a single process, it suffers from state contamination across test cases, leading to incorrectness and missed bugs. Conversely, approaches like fork-server are correct but incur significant process management overhead.

The core contribution of CLOSUREX is a novel point on this "state restoration continuum." By using a series of LLVM passes, CLOSUREX instruments a target program to become "self-resetting." It injects a persistent fuzzing loop and automatically adds code to track and roll back key sources of program state—specifically global variables, heap allocations, and file descriptors—between each test case. This approach effectively simulates a fresh process for each input while eliminating the overhead of process creation and destruction. The evaluation demonstrates that CLOSUREX achieves a ~3.5x performance increase over AFL++'s standard fork-server mode and finds bugs 1.9x faster, all while maintaining the semantic correctness of fresh-process execution.

Strengths

The true strength of this work lies in its elegant and highly practical approach to a long-standing, important problem in the fuzzing community.

1. Excellent Problem Formulation and Positioning: The authors do a superb job of contextualizing their work. The continuum from fresh-process (correct, slow) to persistent (fast, incorrect) is a perfect framing. Table 1, in particular, is a masterful piece of communication, immediately showing the gap in the design space that CLOSUREX aims to fill: a portable, correct, high-performance solution that works for whole applications.

2. A Pragmatic and Portable Solution: By choosing a compiler-based approach (LLVM), the authors sidestep the major pitfalls of competing high-performance solutions. Unlike kernel-based snapshotting (e.g., [7, 34]), CLOSUREX is OS-agnostic and does not rely on fragile, version-specific kernel interfaces. This makes the solution far more deployable and maintainable. It represents a user-space, compile-time alternative to both kernel-level and binary-level snapshotting techniques [29], occupying a sweet spot of performance and accessibility.

3. Addresses the "Annoying Last Mile" of Persistent Fuzzing: While experts have long known how to write manual reset functions for persistent mode, this process is tedious, error-prone, and requires deep target-specific knowledge. CLOSUREX automates this harness generation, democratizing high-performance, correct fuzzing. This is a significant engineering contribution that lowers the barrier to entry for effective fuzzing campaigns.

4. Strong Empirical Validation: The performance gains are substantial and well-documented across a diverse set of standard fuzzing benchmarks. The most critical part of the evaluation, the correctness check in Section 6.5, is well-designed. By verifying both dataflow and control-flow equivalence against a ground-truth fresh-process execution, the authors provide strong evidence for their central claim of maintaining semantic correctness.

Weaknesses

The weaknesses of the paper are primarily related to the boundaries and limitations of the proposed technique, which could be explored more deeply.

1. Scope of State Restoration: The current implementation handles the most common sources of state (globals, heap, file descriptors). However, real-world applications often involve more complex state. The discussion in Section 7.4 acknowledges this, but the paper would be stronger if it more formally defined the classes of state it cannot handle. For example, state hidden in linked, non-instrumented libraries, state managed by custom allocators, interactions with hardware, or persistent changes to the filesystem are all outside the current scope. Similarly, the proposed method for handling threads in Section 7.3 seems optimistic; managing thread-local storage and ensuring clean thread teardown is notoriously difficult.

2. The Challenge of Initialization Overhead: The paper's primary performance comparison is against the fork-server. A key advantage of the fork-server is that it snapshots the process after program initialization. CLOSUREX, by looping around main, re-executes this initialization code for every single test case. For programs with a heavy, one-time setup cost, this could significantly erode the performance benefits. The authors acknowledge this as future work (Section 7.1), but it remains a notable limitation of the current approach when compared to the fork-server model.

3. Lack of Comparison to Expert-Crafted Harnesses: The performance evaluation is missing a key baseline: a manually-written, expert-crafted persistent mode harness for one of the simpler targets (like zlib). While the value of CLOSUREX is its automation, understanding the performance overhead of its generic state-tracking mechanisms compared to a bespoke, minimal reset function would provide valuable context. Is there a performance price for this automation?

Questions to Address In Rebuttal

1. Regarding initialization overhead: Could you provide an estimate or measurement for one of your benchmarks (e.g., bsdtar) of the proportion of execution time spent in one-time initialization code that CLOSUREX re-executes but a fork-server would not? This would help clarify the types of targets where CLOSUREX offers the greatest benefit.

2. Regarding complex state: How would the CLOSUREX model handle C++ programs, specifically with respect to static object constructors/destructors and exceptions? Does the setjmp/longjmp mechanism for replacing exit() correctly unwind C++ objects, or could it lead to resource leaks?

3. The proposed solution for resetting heap state by hooking malloc/free is elegant. However, many complex C/C++ applications use custom memory allocators (e.g., arena or slab allocators) for performance. How would a user adapt CLOSUREX to support a target with a custom allocator that manages a large, contiguous block of memory internally? Would this require manual annotation or a new set of passes?

Reviewer: The Innovator (Novelty Specialist)

Summary

This paper presents CLOSUREX, a system for achieving high-performance, correct persistent fuzzing. The authors' core claim is a method for transforming standard C programs into "naturally restartable" ones using a series of compile-time LLVM passes. This instrumentation automatically injects code to track and reset key sources of program state—specifically global variables, the heap (via malloc/free hooks), the stack (via setjmp/longjmp), and file descriptors (via fopen/fclose hooks)—between fuzzing iterations within a single process. This avoids the overhead of process creation (fork/exec) while maintaining the semantic correctness lost in traditional persistent fuzzing modes.

Strengths

The primary strength of this paper lies in its specific implementation strategy. The application of compiler-level instrumentation to automate state-reset for persistent fuzzing is an elegant engineering approach. By operating at the LLVM IR level, the authors have a principled way to intercept state-modifying library calls and manage memory sections. This source-based approach is a clean alternative to OS-level primitives or binary-level rewriting and offers the potential for fine-grained, precise state management.

Weaknesses

The central weakness of this paper, from a novelty perspective, is that the core conceptual idea—in-process state-saving and restoration to enable fast, correct fuzzing—is not new. The work is best characterized as a novel implementation of a pre-existing concept.

1. Overlap with Prior Art: The work is conceptually very similar to WinFuzz (Stone et al., NDSS '21) [11] and its follow-on work (Stone et al., USENIX Security '23) [29]. WinFuzz also implements in-process snapshotting and restoration to bypass OS-level overhead. It achieves this by rewriting the binary to save and restore memory regions (heap, stack, globals) and other process state. While the mechanism differs (LLVM instrumentation vs. binary rewriting), the fundamental idea of creating a self-resetting process for fuzzing is the same. The authors acknowledge this work in Section 8.2 (Page 12) but claim their approach is superior due to being "fine-grain" and avoiding "runtime code injection overhead." However, this claim is presented qualitatively and is not substantiated with a direct comparison, making the "delta" over prior art seem incremental—an alternative engineering choice rather than a new paradigm.

2. Limited Scope of State Restoration: The novelty is further constrained by the specific types of state handled. The presented solution hooks standard libc functions (malloc, fopen, exit). This approach is well-understood but does not address more complex, yet common, scenarios. For instance, programs employing custom memory allocators, memory-mapped files (mmap), direct syscalls for I/O, or state stored in shared memory would not be correctly reset by CLOSUREX out-of-the-box. The authors acknowledge this in Section 7.4 (Page 11), but this limitation implies that the "automatic" solution is only automatic for a well-behaved subset of programs. The novelty is thus in the specific implementation of these hooks, not in a generalizable state-reset framework.

3. Well-Known Techniques: The techniques used for state restoration are, individually, not novel. Using wrappers around malloc/free to track allocations is a standard technique used in memory debuggers for decades. The use of setjmp/longjmp to hijack control flow from exit() is a classic C programming idiom for implementing exception-like behavior. The novelty lies only in the composition of these specific techniques for the fuzzing use case.

Questions to Address In Rebuttal

1. Clarify the Delta vs. WinFuzz [29]: The authors claim that binary-level snapshotting (as in WinFuzz) is subject to "imprecision with its state checkpoints." Please provide a concrete example of a state-related bug or inconsistency that would be missed by a binary-level approach like WinFuzz but correctly handled by CLOSUREX's compiler-level instrumentation. What, precisely, is this "imprecision"?

2. Generalizability of the Technique: How much manual, target-specific effort would be required to adapt CLOSUREX to a target that uses a custom slab allocator instead of malloc? If the technique relies on developers manually identifying and hooking all sources of latent state, how does this represent a significant leap over the manual reset handlers already used in tools like libFuzzer?

3. Robustness of longjmp: The use of longjmp to unwind from a call to exit() is a C-style mechanism. Have the authors considered its correctness in C++ programs, where this would bypass the execution of destructors for stack-allocated objects, potentially leading to resource leaks or incorrect state for the next fuzzing iteration? This could undermine the core claim of "semantic correctness."

Paper Title: Coach: Exploiting Temporal Patterns for All-Resource Oversubscription in Cloud Platforms Reviewer: The Guardian

---

Summary

The authors propose "Coach," a system for oversubscribing all major resources (CPU, memory, network, etc.) in a cloud environment by exploiting temporal utilization patterns. The core mechanism is a new VM type, the "CoachVM," which partitions each resource into a "guaranteed" portion, backed by physically allocated resources (e.g., PA-backed memory), and an "oversubscribed" portion, backed by a shared pool (e.g., VA-backed memory). The system relies on a prediction model to forecast utilization across daily time windows, enabling a scheduling policy that co-locates VMs with complementary usage patterns. The authors claim this approach can increase platform capacity by up to ~26% with minimal performance degradation.

Strengths

1. The characterization study presented in Section 2 is thorough and provides a solid motivation for the work. The analysis correctly identifies that oversubscribing a single resource like CPU simply shifts the bottleneck to other resources (e.g., memory), as shown in Figures 4 and 5. This effectively builds the case for a holistic, all-resource approach.

2. The fundamental design of the CoachVM (Section 3.2), which separates guaranteed and oversubscribed resource allocations, is a practical and logical construct. Using physically-backed (PA) memory for the guaranteed portion and virtually-backed (VA) memory with zNUMA for the oversubscribed portion is a sound mechanism for attempting to isolate performance-critical working sets from reclamation pressure.

3. The evaluation is commendably broad in its scope, attempting to address single-VM performance on real hardware (Section 4.2), scheduling policy effectiveness at scale via simulation (Section 4.3), and the efficacy of contention mitigation policies (Section 4.4).

Weaknesses

My primary concerns with this paper center on the fragility of its core assumptions, a disjointed evaluation that fails to connect its key components, and an underestimation of the severity of contention events.

1. Extreme Sensitivity to Prediction Error: The entire system's safety and performance guarantees rest on the ability to accurately predict a VM's working set to establish the PA/VA memory split. The paper's own results demonstrate that this is a knife-edge problem. In Figure 18, the "CVM-Floor" configuration, which emulates a mere 1GB under-allocation of the guaranteed portion, results in a catastrophic performance degradation of up to 1.8x for sensitive workloads like KV-Store. This indicates the system has no safety margin. While the authors claim their predictor is accurate (Figure 19), the grouping analysis in Figure 12 (p. 6) shows that even for the best grouping strategy, the median VM has a utilization range of 31% for memory. This is an enormous variance, and it is the tail of this distribution—the unpredictable VMs—that will cause cascading failures. The paper fails to adequately prove that its prediction model is robust enough to prevent these frequent and severe performance cliffs in a real-world, large-scale deployment.

2. Disconnected and Unvalidated Evaluation Methodology: The evaluation is critically fractured. The authors evaluate single-VM performance on a physical server (Section 4.2) and scheduling policy savings in a large-scale simulation (Section 4.3). However, there is no end-to-end evaluation that validates whether the scheduling decisions made by the simulator lead to the acceptable performance outcomes measured on the physical server. The simulation's model of contention is particularly suspect. In Section 4.3, the authors state that "memory contention occurs when memory accesses result in page faults" and show "performance violations" in Figure 20b. This is not a performance model; it is a binary event counter. How does the simulation model the non-linear, system-wide performance impact of page faults, increased I/O pressure, and CPU scheduler contention that would occur when thousands of CoachVMs are co-located? Without this crucial link, the simulation results on capacity savings are purely theoretical and cannot be trusted to reflect a production reality.

3. The Severity of Contention is Understated: The mitigation analysis in Section 4.4, while interesting, paints a far rosier picture than the data suggests. Figure 21 shows that under memory pressure, workload performance degrades by up to 4.3x before the mitigation policy fully resolves the issue. The paper claims its proactive policies "reduce this overhead to only 1.3x" (p. 15), but this is a relative improvement on a catastrophic event, not a guarantee of acceptable performance. The x-axis of Figure 21 is in seconds. For a latency-sensitive service, a multi-second window of 4.3x (or even 1.3x) higher latency is not "minimal performance degradation"; it is a severe SLO violation and a functional outage. The paper does not analyze the frequency and duration of these contention events at scale.

4. The "All-Resource" Claim is Not Substantiated: The paper's title promises an "All-Resource Oversubscription" system, but the design and evaluation are overwhelmingly focused on CPU and memory. Other critical and non-fungible resources, such as local SSD IOPS/bandwidth and network bandwidth for SR-IOV-enabled VMs, receive only cursory mention. The challenges of oversubscribing these resources (e.g., I/O interference in the storage controller, NIC scheduler contention) are non-trivial and fundamentally different from memory paging. The paper does not present a credible design or evaluation for how Coach would manage contention for these resources, thereby failing to deliver on its primary claim.

Questions to Address In Rebuttal

1. Given the extreme performance penalty for a minor misprediction (1.8x slowdown for a 1GB error, Figure 18), how does the system defend against correlated prediction errors, such as an unexpected, region-wide event (e.g., breaking news, service discovery failure) causing thousands of co-located VMs to simultaneously expand their working sets? What is the expected rate of SLO violations under such a "black swan" scenario?

2. Please provide a detailed description of the performance model used in the large-scale simulator to translate resource over-allocation events into the "Performance violations" metric in Figure 20b. How was this model calibrated and validated against the performance of real, co-located workloads under contention on physical hardware?

3. The mitigation experiments (Figure 21) show performance degradation lasting for several seconds. In a production environment, what is the expected distribution of contention event durations (from onset to full mitigation), and what percentage of VMs in a cluster are expected to be experiencing such a degradation event at any given time?

4. Regarding the "new VM" problem: What specific mechanism and default PA/VA ratio does Coach use for a VM from a new customer subscription or a new application configuration with no historical data? How does this conservative default impact the ~26% capacity savings claim, as these VMs would presumably be unable to participate fully in oversubscription?

Paper: Coach: Exploiting Temporal Patterns for All-Resource Oversubscription in Cloud Platforms Reviewer: The Synthesizer (Contextual Analyst)

---

Summary

This paper presents Coach, a system designed to improve resource utilization in large-scale cloud platforms by enabling holistic, all-resource oversubscription. The core contribution is the synthesis of three key ideas: (1) a comprehensive characterization of production VM workloads that reveals predictable, complementary temporal (e.g., diurnal) patterns of resource usage; (2) a new VM abstraction, the CoachVM, which partitions each resource allocation into a "guaranteed" portion for performance stability and an "oversubscribed" portion for efficiency; and (3) a time-window-based predictive scheduling policy that colocates VMs with complementary usage patterns to maximize server density safely.

The authors focus significantly on the challenges of memory oversubscription, a notoriously difficult problem in virtualized environments, proposing a practical PA/VA-backing solution. Through extensive simulation and workload-based experiments, they demonstrate that Coach can increase the number of hosted VMs by up to ~26% with minimal performance degradation, addressing a problem of immense economic importance to cloud providers.

Strengths

1. Addresses a Fundamental Problem with a Holistic View: The problem of low resource utilization in datacenters is well-established, but much of the prior art has focused on CPU oversubscription (e.g., harvesting). The key strength of this paper is its holistic approach. The characterization study in Section 2 is compelling, clearly demonstrating that a CPU-only solution merely shifts the bottleneck to other resources like memory and network. By designing a system that considers all resources, Coach provides a much more complete and practical solution for modern cloud platforms.

2. Excellent Synthesis of Existing Concepts: This work stands out for its successful synthesis of ideas from cluster scheduling, workload prediction, and virtual memory management. While temporal analysis and oversubscription are not new in isolation, their combination within a cohesive system for virtualized environments is novel and powerful. The paper effectively builds upon the lineage of large-scale cluster managers like Borg [96] but adapts the principles to the unique and more challenging context of opaque, multi-tenant VMs rather than containers.

3. The CoachVM as a Practical Abstraction: The introduction of the CoachVM (Section 3.2, page 7) is a significant practical contribution. It provides a clean abstraction for both the cloud provider and potentially the customer. The separation of "guaranteed" and "oversubscribed" resources is an elegant way to manage the fundamental trade-off between performance isolation and resource efficiency. The detailed discussion of handling non-fungible resources like memory, including the PA/VA split and considerations for DMA/SR-IOV, shows a deep understanding of the real-world systems challenges involved.

4. Strong Empirical Grounding: The work is built on a solid foundation of data. The initial characterization study on over one million production VMs in Azure (Section 2) provides a strong motivation and is a valuable contribution in its own right. The evaluation (Section 4) is thorough, wisely using both large-scale simulation to assess packing gains and real-world experiments with diverse benchmarks to quantify the performance impact of contention and the effectiveness of mitigation strategies.

Weaknesses

While this is a strong paper, there are opportunities to further contextualize the work and explore the boundaries of the proposed approach.

1. The Inevitable Complexity of Memory Management: The paper commendably tackles memory oversubscription head-on. However, the proposed solutions, particularly the need for "guest enlightenments" (paravirtualization) to handle legacy devices without ATS/PRI support (Section 3.2, page 8), represent a slight departure from the ideal of a fully transparent solution. While this is a pragmatic engineering choice, it highlights a tension in the system's design goals and points to the inherent difficulty of managing memory without some level of guest cooperation or advanced hardware support.

2. Implicit Assumption of Stable Macro Patterns: The system's effectiveness hinges on the existence and predictability of complementary temporal patterns. While the characterization study confirms their current existence, the work doesn't deeply explore the potential for these patterns to change over time, especially in response to the system itself. For instance, if CoachVMs are offered at a discount, might that incentivize customers to shift workloads, thereby eroding the very complementarity that Coach exploits? A discussion of this potential feedback loop would add depth.

3. Positioning Relative to Container Orchestration: The paper correctly identifies the unique challenges of VMs. However, it could more explicitly articulate why a state-of-the-art container-based approach (e.g., Borg, Twine) is insufficient for the IaaS cloud use case. Drawing a sharper contrast would help readers from the container world better appreciate the specific contributions required for virtualized environments.

Questions to Address In Rebuttal

1. The PA/VA ratio for a CoachVM seems critical to balancing performance and savings. The paper describes the trade-off in Figure 15 (page 8) but is less explicit about the policy for setting this ratio in practice. How is the guaranteed (PA-backed) portion for a new VM determined? Is it based on a fixed percentile (e.g., P95 of historical usage for similar VMs), or is it a more dynamic policy?

2. Could you elaborate on the potential for a systemic feedback loop? If Coach is widely deployed and customers adapt their behavior to its associated pricing models (e.g., cheaper off-peak CoachVMs), how might this affect the stability of the complementary patterns you observed? Does the system have mechanisms to adapt to such long-term shifts in aggregate user behavior?

3. Regarding the mitigation policies (Section 4.4, page 14), live migration is presented as a last resort. Given that CoachVMs are designed to exploit predictable, long-term patterns, have you considered proactive, slow "rebalancing" migrations during off-peak hours to optimize a server's mix of VMs, rather than relying solely on reactive migration during contention events? This seems like a natural extension of leveraging temporal knowledge.

Paper Title: Coach: Exploiting Temporal Patterns for All-Resource Oversubscription in Cloud Platforms Reviewer Persona: The Innovator (Novelty Specialist)

---

Summary

This paper, "Coach," proposes a system for holistic, all-resource oversubscription in cloud platforms by exploiting complementary temporal utilization patterns of VMs. The authors' characterization of production traces from Azure reveals that many VMs have predictable, out-of-phase resource peaks. To leverage this, Coach introduces a time-window-based scheduling policy that uses a prediction model to forecast VM resource needs across different times of day. This allows for more aggressive and intelligent co-location of VMs. The system is built around a new VM type, "CoachVM," which partitions resources into a "guaranteed" portion (backed by physically allocated resources like PA memory) and an "oversubscribed" portion (backed by a shared pool, e.g., VA memory). The evaluation, based on simulations and workload benchmarks, shows that Coach can host up to ~26% more VMs compared to a baseline oversubscription policy.

Strengths

The paper presents a comprehensive system design and a large-scale evaluation based on production traces from a major cloud provider. The proposed synthesis of temporal pattern prediction, a multi-resource scheduler, and a new VM abstraction (CoachVM) into a single cohesive system is a strength. The characterization study in Section 2 is thorough and provides a strong motivation for the work.

Weaknesses

My review focuses exclusively on the novelty of the core ideas presented. While the engineering and integration are substantial, the fundamental concepts underpinning Coach appear to be largely derived from prior art.

1. Core Idea of Exploiting Temporal Patterns is Not New: The central thesis—that workloads exhibit complementary temporal patterns (e.g., diurnal cycles) and can be co-located to improve utilization—is a well-established concept in datacenter management. Over a decade ago, Chen and Shen proposed consolidating "complementary VMs with spatial/temporal-awareness" [20]. Their work identified the same opportunity and proposed a similar solution of pairing VMs with anti-correlated resource usage patterns. Coach appears to be a modern, large-scale, and more sophisticated implementation of this foundational idea, but the core conceptual leap is not present. The "time window" mechanism described in Section 3.3 (page 9) is an implementation detail for a known principle.

2. The "CoachVM" Abstraction is an Amalgamation of Existing Concepts: The proposal of a VM with a guaranteed baseline and a burstable/oversubscribed portion is functionally identical to existing commercial offerings, such as AWS's Burstable Performance Instances (T-series) and Azure's own B-series VMs [8]. The novelty cannot be claimed for the abstraction itself. Furthermore, the primary technical mechanism detailed for implementing this for memory—partitioning into a PA-backed guaranteed portion and a VA-backed oversubscribed portion using zNUMA—is a direct application of the technique described in the Pond paper [54] from many of the same authors. Pond introduced CXL-based memory pooling with zNUMA to abstract remote memory; Coach applies the same underlying OS/hypervisor mechanism for oversubscription. The novelty is in the application of this mechanism, not the mechanism itself.

3. Holistic "All-Resource" Management is Conceptually Preceded by Container Orchestrators: The claim of novelty in "all-resource" oversubscription must be considered in the context of large-scale cluster managers like Google's Borg [94, 96] and Facebook/Meta's Twine [93]. These systems have long managed complex, multi-resource (CPU, RAM, disk I/O) oversubscription for containers. While the paper correctly identifies that VMs present a more difficult, opaque environment, the conceptual framework for multi-resource bin-packing and managing contention is not new. Twine, for example, explicitly orchestrates containers with user-requested CPU or memory oversubscription. The primary delta here is the target domain (VMs vs. containers), which is an important engineering distinction but a small conceptual one. The paper's technical deep dive (Sections 3.2, 3.4) also focuses almost entirely on memory, which weakens the claim of a novel, truly "all-resource" contribution.

In summary, the contribution of this paper appears to be one of significant systems engineering and integration, rather than fundamental innovation. It combines the known idea of temporal-aware scheduling [20] with the known mechanism of PA/VA memory partitioning [54] and applies it to the VM domain, which has been conceptually addressed in the container domain [93, 96]. The "delta" over the prior art is the specific synthesis and large-scale validation in a production VM environment, which is valuable but incrementally novel.

Questions to Address In Rebuttal

1. How does the core idea of exploiting complementary temporal patterns in 'Coach' fundamentally differ from the temporal-aware VM consolidation proposed in Chen and Shen, INFOCOM 2014 [20]? Please be specific about the conceptual novelty beyond scale and implementation choices.

2. The CoachVM's memory model (PA-guaranteed, VA-oversubscribed using zNUMA) appears to be a direct application of the mechanism from Pond, ASPLOS 2023 [54]. Could the authors clarify the novel technical contribution in this mechanism beyond its application to an oversubscription policy?

3. While the paper claims "all-resource" oversubscription, the deep dive focuses almost exclusively on memory. Could the authors elaborate on the novel mechanisms developed for handling the non-fungibility and unique contention characteristics of other resources like network I/O or local SSD IOPS, and how these mechanisms advance the state of the art?

Paper Title: Composing Distributed Computations Through Task and Kernel Fusion Reviewer: The Guardian

---

Summary

The authors present Diffuse, a system that sits between high-level distributed libraries (cuPyNumeric, Legate Sparse) and a low-level task-based runtime (Legion). The core contribution is a "scale-free" intermediate representation (IR) designed to enable scalable analysis for task fusion in a distributed memory setting. This task fusion is paired with an MLIR-based JIT compiler to perform kernel fusion on the fused task bodies. The authors claim that this approach can accelerate unmodified applications by a geometric mean of 1.86x on up to 128 GPUs, and in some cases, match or exceed the performance of a manually-tuned MPI library, PETSc.

Strengths

1. The fundamental concept of a middle-layer, domain-agnostic IR for identifying fusion opportunities across library boundaries is a sound and valuable direction for research in composable software.
2. The IR's "scale-free" design, which makes the cost of analysis independent of the machine size, is an elegant solution to a known scalability challenge in distributed program analysis.
3. The integration with MLIR for the kernel fusion backend is a practical and powerful choice, allowing the system to leverage a robust, community-driven compiler infrastructure.
4. The system is demonstrated on two distinct, real-world libraries, providing some evidence for the generality of the proposed IR and analyses.

Weaknesses

My primary responsibility is to ensure the technical and empirical soundness of papers accepted by this conference. While the ideas presented are interesting, I have significant concerns about the methodology and the strength of the evidence supporting the paper's core claims.

1. Conflation of Contributions due to Missing Ablation Study: The paper claims end-to-end speedups but fails to disentangle the sources of performance gain. The total speedup comes from a combination of: (1) task fusion reducing runtime scheduling overhead, (2) temporary store elimination reducing memory traffic and allocation, and (3) kernel fusion improving data locality and arithmetic intensity. The latter two are well-known, powerful optimizations. The novel contribution is the scale-free IR enabling task fusion. Without an ablation study that evaluates "task fusion only" vs. "task fusion + kernel fusion", it is impossible to assess the actual benefit of the paper's core idea. It is plausible that most of the performance gain stems from standard MLIR-based loop fusion (a known technique), with the task fusion component providing only marginal benefit by enabling it. The authors explicitly state, "We do not ablate on the optimizations in Section 5" (page 10), which is a critical methodological flaw.

2. Questionable Baseline Comparisons: The comparison against PETSc is presented as a major result, but the experimental setup is unsound. A footnote on page 10 reveals that Legate Sparse was modified to use 32-bit integers for coordinates to match a specific behavior of PETSc. This is not a fair comparison. A rigorous evaluation would require ensuring both systems are configured optimally and use identical data types and precision for the computation. As presented, the authors are comparing a version of their system specifically tuned for the benchmark against what may be a default, unoptimized, or misconfigured baseline. This undermines the claim of "exceeding the performance of PETSc."

3. Limited Expressiveness of the IR: The paper's claims of generality hinge on its IR. However, Section 3 only formally describes None and Tiling partitions. These structured, affine partitions are well-suited for dense computations but are insufficient for a wide range of scientific applications involving irregular data structures, such as unstructured meshes or graph analytics, which require more complex, indirect partitioning schemes. The authors state their "implementation supports more partition kinds" (page 4) but provide no details. Without this information, the reader cannot evaluate the true generality of the approach. The constant-time alias checking, a key enabler of the scalable analysis, is a direct consequence of this restricted representational power.

4. Benchmark Suite Bias Skews Aggregate Results: The reported geometric mean speedup of 1.86x is heavily distorted by the Black-Scholes benchmark, which achieves a 10.7x speedup (Figure 10a). This application is a textbook example of an embarrassingly parallel computation with a long chain of fusible operations—a "hero" benchmark that represents the absolute best-case scenario for this system. In contrast, the Dense Jacobi benchmark shows no meaningful improvement (0.93x-1.08x), demonstrating the system's limitations. Presenting a single geometric mean obscures this reality and overstates the typical performance gain a user should expect.

5. Insufficient Proof of Correctness: The proof sketch for the fusion algorithm's correctness (Section 4.3) is cursory. It argues that the fusion constraints are sufficient to guarantee point-wise dependencies but does not provide a formal argument. Furthermore, the paper acknowledges the constraints are "sound, but not complete" (page 6). There is no discussion of the practical implications of this incompleteness. What types of valid fusion opportunities are missed by this conservative analysis? A more thorough treatment is necessary to establish confidence in the algorithm's robustness and to understand its limitations.

Questions to Address In Rebuttal

The authors must address the following points to convince me of the paper's validity:

1. Can you provide an ablation study that isolates the performance impact of task fusion (i.e., runtime overhead reduction) from the impact of kernel fusion and temporary elimination? This is essential to substantiate the value of your primary contribution.
2. Please justify the fairness of the PETSc comparison. Ideally, provide results where both PETSc and Diffuse/Legate Sparse are configured to use the same data precision (e.g., both using 64-bit integers for coordinates) and are otherwise optimally tuned.
3. What specific, non-affine partition kinds does your system support beyond the Tiling described in the paper? How does your scale-free alias analysis handle these more complex, potentially irregular partitions?
4. Can you provide a performance breakdown that clarifies the impact of outliers? For example, what is the geometric mean speedup if the Black-Scholes benchmark is excluded?
5. What is a concrete example of a valid fusion that your constraints (Figure 5) would incorrectly disallow? A discussion of the trade-off between analytical complexity and optimization power is needed.

Paper: Composing Distributed Computations Through Task and Kernel Fusion Reviewer: The Synthesizer (Contextual Analyst)

---

Summary

This paper introduces Diffuse, a system that sits between high-level task-based parallel libraries and a low-level distributed runtime system to perform dynamic optimization. The core contribution is a novel, scale-free intermediate representation (IR) for distributed computations. This IR is the key enabler for the paper's primary goal: to scalably analyze and fuse streams of distributed tasks, even when those tasks originate from different libraries. By identifying fusible sequences of tasks, Diffuse not only reduces runtime overhead but also creates opportunities for a JIT compiler (built on MLIR) to perform aggressive kernel fusion, eliminating temporary distributed allocations and improving data locality. The authors demonstrate Diffuse's effectiveness by integrating it with the cuPyNumeric and Legate Sparse libraries running on the Legion runtime, showing significant performance improvements (1.86x geo-mean) on a variety of scientific applications on up to 128 GPUs, often matching or exceeding the performance of hand-optimized code and established MPI-based libraries like PETSc.

Strengths

This is an excellent paper that makes a strong contribution to the field of high-performance, composable software. Its primary strengths are:

1. Elegant Separation of Concerns: The system's architecture is its most powerful feature. By introducing a "middle layer" with a purpose-built IR, the authors cleanly decouple the problem of distributed dependency analysis from the problem of single-node kernel optimization. The scale-free IR (Section 3, Page 3) is designed to make the former tractable and scalable, while the integration with MLIR (Section 6, Page 8) leverages a powerful, existing ecosystem for the latter. This is a very insightful way to structure the problem.

2. The Scale-Free IR as a Core Contribution: The central idea of a "scale-free" representation—one whose complexity is independent of the machine size—is critical. It directly addresses the primary challenge of performing program analysis on massively parallel systems, where reasoning about every concrete task instance is infeasible. This approach builds philosophically on concepts like Legion's Index Launches [50], but applies the principle specifically to the problem of dynamic, cross-library fusion. The ability to perform constant-time alias checks on structured partitions is a direct and valuable result of this IR design.

3. Tackling the Composability "Holy Grail": Many systems have attempted to optimize across library boundaries, but most, like Weld [44], have focused on shared-memory contexts. A key contribution of this work is demonstrating a practical path forward for composition in the more complex distributed memory setting. By operating on a common task-based abstraction, Diffuse successfully finds optimization opportunities that are invisible to individual libraries, as shown in the evaluation where it improves upon already hand-optimized code (Section 7.1, Page 11, Figure 12c).

4. Impressive and Convincing Empirical Results: The evaluation presented in Section 7 is comprehensive and compelling. The authors use a weak-scaling methodology, compare against strong and relevant baselines (including hand-tuned versions and the highly-respected PETSc library), and evaluate a range of applications from micro-benchmarks to full-fledged scientific solvers. The results, showing consistent and significant speedups without any application code changes, strongly validate the system's design and practical utility.

Weaknesses

The paper is very strong, and the weaknesses are more about the boundaries of the current work and opportunities for future exploration rather than fundamental flaws.

1. Uncertainty on Generality of the Data Model: The paper's IR for partitions is presented with a focus on structured kinds like Tiling (Section 3.1, Page 4). While the authors state their implementation supports more, the power of underlying runtimes like Legion comes from their ability to handle complex, irregular, and data-dependent partitions. It is not entirely clear how the scale-free, symbolic analysis proposed by Diffuse would extend to these more unstructured cases, where checking for aliasing may no longer be a simple, constant-time symbolic operation. This might represent a fundamental tension between the analyzability of the IR and the expressiveness of the underlying runtime.

2. Limited Scope of Fusion Strategy: The fusion algorithm is greedy and focuses on finding a fusible prefix of tasks in a linear window (Section 4.2.2, Page 6). This is a pragmatic and effective starting point. However, this places the work firmly in the category of local, peephole-style optimizations. More global optimization opportunities, such as reordering independent tasks to create larger fusible blocks or fusing non-contiguous tasks, are not considered. While out of scope for one paper, a discussion of the limitations of the current strategy would be welcome.

3. Potential for Adverse Interaction with Runtime Schedulers: Diffuse fundamentally alters the task graph, transforming many small tasks into fewer, larger ones. This has implications for the downstream runtime scheduler. For example, a very long fused task could negatively impact load balancing or increase tail latency. The paper does not explore this interaction. While the performance results suggest this is not a problem for the benchmarks chosen, it remains a potential issue for more dynamic or heterogeneous workloads that could be addressed.

Questions to Address In Rebuttal

1. Regarding the partition model in the IR (Section 3.1): Could you please elaborate on the challenges of extending your scale-free analysis to the more general, unstructured partitions supported by Legion? Specifically, how would the system perform alias analysis on partitions defined by arbitrary collections of points, and what would be the impact on the scalability of the analysis?

2. The current fusion algorithm is greedy and limited to contiguous task prefixes. Could you comment on the potential for more sophisticated fusion strategies within the Diffuse framework? For example, what would be the primary obstacles in the IR or analysis to support reordering tasks to enable larger fusions?

3. Your system transforms the task stream by creating fewer, larger-grained tasks. How does this transformation affect the scheduling and load-balancing capabilities of the underlying runtime (e.g., Legion)? Have you observed any cases where creating a very large fused task leads to performance degradation due to, for instance, poor work distribution?

Reviewer: The Innovator

Summary

This paper introduces Diffuse, a system for performing dynamic task and kernel fusion for applications built on distributed, task-based runtimes. The central claim of novelty rests on a new intermediate representation (IR) for distributed computation. This IR is described as "scale-free," meaning its size and the complexity of analyses performed upon it are independent of the number of processors in the target system. The authors leverage this IR to perform a scalable, dynamic dependence analysis that identifies fusible sequences of distributed tasks. This task fusion is then coupled with a JIT compiler (built on MLIR) that fuses the kernels within the newly formed tasks. The authors claim this synthesis of scalable distributed task fusion and kernel fusion enables optimization across library boundaries, allowing high-level Python programs to match or exceed the performance of hand-tuned MPI code.

Strengths

The primary strength and novel contribution of this work is the design of the scale-free IR (Section 3, page 3) and the fusion analysis framework built upon it. While the constituent ideas—task fusion, kernel fusion, JIT compilation—are not new in isolation, their synthesis in this specific context is. The key insight is that by creating a higher-level, more structured representation of distributed partitions and computations than what a low-level runtime like Legion provides, certain critical analyses (like aliasing) become tractable at scale.

Specifically, the novel aspects are:

1. The Scale-Free IR Abstraction: The formalization of distributed data and computation into Store, structured Partition types (e.g., Tiling), and Index Task constructs is the core technical kernel. This abstraction enables symbolic, constant-time alias checking between structured partitions, which is the cornerstone of the scalable fusion analysis. This is a clear advancement over analyzing the lower-level, unstructured partitions that a runtime like Legion might expose, where such a check would scale with the number of processors.

2. A Formal Framework for Distributed Task Fusion: Section 4 (page 5) presents a formal definition of dependencies between distributed task groups via a "dependence map." The paper then develops a set of fusion constraints (Figure 5, page 6) that can prove the non-existence of cross-processor dependencies without needing to materialize this map. While the dependency types (true, anti, reduction) are classic compiler concepts, their formulation and application within this scale-free, distributed IR are novel.

3. Synthesis of Two Fusion Levels: The most significant conceptual advance is the coupling of the high-level, distributed task fusion with low-level kernel fusion. Prior work has often focused on one or the other. Diffuse uses the high-level analysis to solve the distributed data-movement and dependency problem, creating a valid fused task. This fused task then presents a traditional, single-node loop fusion problem to the MLIR-based JIT, which can leverage a rich ecosystem of existing compiler techniques. This two-level approach elegantly separates distributed systems concerns from compiler optimization concerns.

Weaknesses

The main weakness of the paper, from a novelty perspective, is its close relationship to prior art from the same research group, which could be made more explicit.

1. Incremental Advance over Prior Art: The paper correctly identifies Sundram et al. [51] as the "most related work." Indeed, that work introduced the core problem of fusing index tasks in a distributed runtime. The current paper is a direct and significant extension of [51]. The delta is the addition of the JIT-based kernel fusion and a more rigorous formalization of the analysis. However, the paper could do a better job of positioning itself by explicitly stating, early on, "Our work builds on Sundram et al. [51] by..." and then enumerating the precise advancements (e.g., the formal model, the proof, and the critical addition of the kernel fusion compiler). As written, the connection is only fully clarified in the related work section, which downplays the incremental nature of the contribution.

2. Limited Expressiveness of the IR Primitives Shown: The paper's exposition of the IR's Partition construct focuses on None and Tiling for simplicity (Section 3, page 4). This structure is what enables the efficient, scale-free analysis. However, a key feature of the underlying Legion runtime is its ability to handle complex, irregular, and data-dependent partitions. It is not clear from the paper how the Diffuse IR and its constant-time alias analysis would extend to these more complex partitionings. If the novel technique is only applicable to dense, affine partitions, its overall novelty and impact are somewhat constrained. The authors mention their implementation supports more, but this is not substantiated in the paper.

3. Novelty of Constraints vs. their Application: The fusion constraints themselves (launch-domain equivalence, true/anti/reduction dependence checks in Figure 5, page 6) are conceptually direct analogues of classic data-dependency checks. The novelty is not in the invention of these constraints, but in their precise formulation and application to the paper's novel IR. The authors should be careful to claim novelty for the latter, not the former.

Questions to Address In Rebuttal

1. Could you please explicitly and concisely list the technical contributions of this paper over Sundram et al. [51]? Specifically, beyond the clear addition of kernel fusion, what aspects of the formal model for task fusion and the fusion constraints are new, and what did they enable that was not possible before?

2. The scalability of your fusion analysis hinges on the structured nature of partitions in your IR. How does your framework handle irregular or data-dependent partitions, which are supported by the underlying Legion runtime? If it requires custom partition-kind-specific rules, does this not compromise the generality of the approach? Can you provide an example of how an alias check between two non-affine partitions would be performed in a scale-free manner?

3. In Section 4.2.1 (page 6), you state your fusion constraints are "sound, but not complete." Can you provide a compelling, realistic example of a sequence of fusible distributed tasks that Diffuse would fail to fuse? Understanding the limitations of the analysis is key to evaluating the scope of the novel contribution.

Paper Title: "Concerto: Automatic Communication Optimization and Scheduling for Large-Scale Deep Learning" Reviewer Persona: The Guardian (Adversarial Skeptic)

---

Summary

The authors present Concerto, a compiler framework aimed at automating communication optimization and scheduling for large-scale distributed deep learning. The core idea is to decouple the parallelization strategy from communication optimization. The paper formulates the scheduling problem as a Resource-Constrained Project Scheduling Problem (RCPSP) and employs an off-the-shelf solver, with a heuristic odd-even method to ensure tractability. For synchronous communication, it introduces an "auto-decomposition" technique to create overlap opportunities, which appears to be an extension of prior work. The authors evaluate Concerto against several state-of-the-art systems, including Megatron-LM, JAX/XLA, DeepSpeed, and Alpa, across various parallelism strategies, claiming to match or outperform them.

Strengths

1. Principled Problem Formulation: Framing communication scheduling as a Resource-Constrained Project Scheduling Problem (RCPSP) is a clean and principled approach. It abstracts away the ad-hoc, hand-tuned scheduling logic found in many existing systems into a well-understood optimization problem.

2. Broad Experimental Scope: The evaluation is commendably broad, covering multiple distinct and complex parallelism schemes: Pipeline-Tensor-Data (PTD), ZeRO, Dynamic Axial Parallelism (DAP), and automatic parallelism. Comparing against specialized, highly-tuned baselines for each of these is a non-trivial effort.

3. Decoupling of Concerns: The stated goal of decoupling the parallel strategy from the communication optimization is a valid and important research direction. If successful, such a system would significantly improve generality and reduce the engineering burden of developing new parallelization techniques.

Weaknesses

My primary concerns with this submission relate to the scalability of the proposed method, the robustness of the underlying models, and the interpretation of the experimental results, which appears to overstate the system's benefits.

1. Questionable Scalability of the Compilation Process: The paper positions itself as a solution for "Large-Scale Deep Learning," yet the evaluation is limited to a maximum of 32 GPUs. The core scheduling method relies on solving an NP-hard RCPSP. The authors propose an "odd-even method" (Section 4.4.2) as a heuristic to make this tractable. However, there are no theoretical guarantees or empirical evidence to suggest this heuristic scales. The compilation time analysis in Figure 14 is for 8 GPUs only. A 30-second-per-round compilation for a relatively small ViT model does not inspire confidence for models with tens of thousands of operators running on hundreds or thousands of GPUs. The paper critically fails to address the compilation overhead at a scale that would justify its title.

2. Overstated and Cherry-Picked Performance Claims: The headline performance numbers appear to be cherry-picked from configurations where the baselines are known to be weak.

   • The "maximum performance improvement of 42.9%" over DeepSpeed (Section 7.1.3, Figure 10) occurs in a non-NVLink, low-GPU-count (4 GPUs) setting. The proposed optimization to overlap the optimizer's final all-gather (Section 7.3) is a valid trick, but it hardly justifies a whole compiler framework and does not represent a fundamental scheduling breakthrough.
   • The comparison against Megatron-LM (Figure 9) shows only marginal gains (1-5%) and even regressions (-1%) in the most relevant high-performance setting (NVLink FP16). The larger gains appear only when the communication-to-computation ratio is high (non-NVLink), a scenario for which Megatron-LM is not primarily tuned. This does not constitute "outperforming" the state-of-the-art but rather exploiting a corner case.
   • The claim of a 34% advantage over JAX/XLA (Section 7.1.2) is caveated by the authors' own admission that JAX/XLA's "bucket and communication balance is suboptimal." Therefore, Concerto is outperforming a specific implementation flaw, not the fundamental capabilities of the XLA compiler stack.

3. Dependence on Brittle Heuristics and "Magic Numbers": The claim of being a fully "automatic" system is undermined by key components of its methodology. The cost model for auto-decomposition (Section 5.3) relies on a slowdown factor alpha, which is "empirically set to 1.2". This is a magic number. There is no sensitivity analysis provided. How does this value hold across different GPU architectures, network interconnects, or operator implementations? This appears to be a form of manual tuning that contradicts the paper's core premise of automation and generality.

4. Insufficient Differentiation from Prior Work: The auto-decomposition technique (Section 5) bears a strong resemblance to the "Google Decomposition" work presented in [47]. The authors claim their novelty lies in a more general "Decomposition Context" that can include operators other than MatMul (Section 5.2). However, the paper provides no clear ablation study or microbenchmark to quantify the specific benefit of this generalization. Without this, it is difficult to assess the incremental contribution over the prior art. The performance gains could largely stem from reimplementing [47], not from the novel aspects of Concerto.

5. Lack of Robustness Analysis: The entire scheduling framework relies on operator timings obtained via profiling. Real-world execution times are noisy and can vary. The paper does not discuss how sensitive its RCPSP solver and the resulting schedule are to inaccuracies or noise in the initial profiling data. A small error in profiling a critical path operator could lead to a highly suboptimal global schedule. This is a critical practical consideration that has been ignored.

Questions to Address In Rebuttal

1. Scalability: Please provide concrete data on compilation times (profiling, decomposition, and scheduling) for a significantly larger scale, e.g., a 100B+ parameter model on at least 128 GPUs. How does the "odd-even" heuristic's solution quality and runtime scale as the number of graph nodes and devices increases?

2. Magic Number Justification: Please provide a sensitivity analysis for the alpha=1.2 parameter used in your decomposition cost model (Section 5.3). How was this value determined, and how much does performance degrade if it is suboptimal (e.g., 1.0 or 1.5)? How can the system claim to be general if it relies on such an empirically tuned constant?

3. Isolating Novelty: Can you provide a direct, head-to-head comparison between Concerto's auto-decomposition and a faithful reimplementation of the method in [47] on the same model? This is necessary to isolate and quantify the real-world performance benefit of your proposed "generalized decomposition context."

4. Performance Claims: Please address the charge of cherry-picking. Instead of highlighting maximums, could you report the geometric mean of the speedups across all tested configurations for each baseline (Megatron-LM, DeepSpeed, etc.)? Furthermore, can you justify why outperforming a baseline outside of its target hardware environment (e.g., Megatron-LM on non-NVLink) is a meaningful demonstration of superiority?

5. Robustness to Profiling Noise: How does your system handle variability in operator execution times? Have you evaluated the stability of the generated schedule when introducing artificial noise (e.g., ±5-10%) into the profiled operator latencies?

Reviewer Persona: Synthesizer

Summary

The authors present Concerto, a compiler framework designed to automate communication optimization and scheduling for large-scale distributed deep learning. The paper identifies a critical and persistent challenge in the field: existing communication optimizations are typically hand-crafted, deeply coupled with specific parallelism strategies (e.g., tensor, data, pipeline), and thus lack generality and programmability.

The core contribution of this work is to reframe this ad-hoc optimization landscape into a more principled and general problem. Concerto makes two key conceptual moves:

1. It formulates the task of overlapping computation and communication as a classic Resource-Constrained Project Scheduling Problem (RCPSP), allowing the use of off-the-shelf solvers to find a near-optimal execution schedule.
2. It introduces an "auto-decomposition" pass that analyzes synchronous communication primitives (like all-reduce in the forward pass) and automatically partitions their dependent computations to create fine-grained overlapping opportunities, expanding the solution space for the scheduler.

By decoupling the choice of parallelism from the mechanics of communication optimization, Concerto aims to be a general-purpose backend that can improve performance across a wide range of models and parallel execution plans, including those generated by auto-parallelism systems. The empirical results demonstrate that this general approach can match or even exceed the performance of highly specialized, manually-tuned systems like Megatron-LM and DeepSpeed.

Strengths

This paper's primary strength is its successful synthesis of ideas from different domains to create a more general and foundational solution to a well-known problem.

1. A Principled Abstraction: The most significant contribution is the formalization of communication scheduling as an RCPSP. This elevates the problem from a collection of clever, workload-specific heuristics to a well-understood, formal optimization problem. By connecting the messy reality of GPU execution graphs to the structured world of operations research, the authors provide a powerful and extensible abstraction. This is a clear step forward from the bespoke scheduling logic hard-coded into existing frameworks.

2. Demonstrated Generality: The paper’s central claim of decoupling and generality is convincingly substantiated by its evaluation (Section 7, page 9). The authors show Concerto delivering benefits across four distinct and important parallelization paradigms: PTD parallelism (Megatron-LM, JAX/XLA), ZeRO-powered data parallelism (DeepSpeed), specialized parallelism (DAP in Evoformer), and fully automated parallelism (Alpa). This is compelling evidence that the underlying abstraction is sound and not merely tuned for one scenario. It successfully positions Concerto as a common optimization layer that the field has been missing.

3. Proactive Optimization via Auto-Decomposition: While scheduling finds the best path within a given graph, the auto-decomposition technique (Section 5, page 6) actively reshapes the graph to create better paths. This is a crucial insight. It addresses the difficult problem of synchronous collectives, which often create hard synchronization barriers. This idea builds conceptually on prior work like Google's decomposition for tensor parallelism (Ref [47]), but generalizes it into an automated compiler pass driven by SPMD propagation rules, making it applicable beyond a single pattern.

Weaknesses

The weaknesses of the paper are largely related to the inherent complexity of the abstraction it proposes and the necessary simplifications made for tractability.

1. Scalability of the RCPSP Formulation: RCPSP is NP-hard. The authors acknowledge this and propose a heuristic odd-even method (Section 4.4.2, page 5) to make the solving process tractable for large graphs. While the compilation time results in Figure 14 (page 13) are promising for the tested scales, it's not clear how this heuristic's solution quality and runtime scale to the truly massive models on the horizon (e.g., mixtures-of-experts with extremely large, complex graphs). The abstraction is elegant, but its practical utility hinges on the scalability of this heuristic solver.

2. Sequential vs. Joint Optimization: The framework treats scheduling and auto-decomposition as two separate, sequential passes. However, the optimal decomposition strategy is likely dependent on the scheduling opportunities it creates, and vice-versa. For instance, decomposing a tensor into 16 chunks might be optimal if there's enough independent work to overlap with, but suboptimal otherwise. The paper identifies this as a limitation for future work (Section 9, page 14), but it's a fundamental one. The current decoupled approach is a heuristic that may leave performance on the table compared to a true joint optimization.

3. Fidelity of the Performance Model: The cost model for auto-decomposition relies on profiling and an empirically-derived slowdown factor, alpha = 1.2 (Section 5.3, page 8). While practical, this ties the model's accuracy to the specific hardware and software environment used for profiling. A more analytical model that accounts for machine characteristics (e.g., memory bandwidth, kernel launch overhead, network latency/bandwidth) would make the framework more robust and adaptable to new hardware without extensive re-profiling.

Questions to Address In Rebuttal

1. Could the authors elaborate on the limitations of the odd-even heuristic for the RCPSP solver? At what graph size or complexity do they anticipate either the compile time becoming prohibitive or the solution quality diverging significantly from the global optimum?

2. Regarding the sequential nature of scheduling and decomposition: Could you provide some intuition on a scenario where this decoupling would lead to a notably suboptimal result? What would be the primary challenges in formulating a joint optimization problem that considers both simultaneously?

3. The cost model for decomposition uses an empirical factor alpha. How sensitive are the final performance results to this value? For instance, if alpha were 1.5 on a different hardware architecture (e.g., one with lower memory bandwidth), how would that impact the decisions made by the auto-decomposition pass and the overall speedup?

Reviewer: The Innovator (Novelty Specialist)

---

Summary

The paper introduces Concerto, a compiler framework for optimizing communication in large-scale deep learning. The authors propose to automate communication scheduling and optimization, which are often manually tuned in existing systems. The core technical contributions claimed as novel are twofold:

1. The formulation of the operator scheduling problem as a Resource-Constrained Project Scheduling Problem (RCPSP), which is then solved using an off-the-shelf ILP solver. To manage the NP-hard complexity, a heuristic based on the odd-even sorting pattern is applied.
2. An "auto-decomposition" mechanism that identifies critical synchronous communication operators and automatically decomposes their surrounding computational operators to create opportunities for computation-communication overlap. This is generalized by leveraging SPMD specifications of operators.

The authors integrate these two techniques into a PyTorch-based compiler stack that aims to decouple the parallelization strategy from the communication optimization, thereby offering generality across different parallelism schemes.

---

Strengths (Novelty-centric Evaluation)

The primary strength of this work lies in its attempt to generalize and automate techniques that have previously been applied in more limited or ad-hoc ways.

1. Generalization of Decomposition for Overlap: The most significant novel contribution is the "auto-decomposition" framework (Section 5, Page 6). Prior work, notably from Google [47], demonstrated the effectiveness of decomposing computation (specifically einsum) to overlap with a dependent all-reduce. However, that approach was largely pattern-specific. Concerto's key innovation is to automate and generalize this. By leveraging SPMD propagation rules (from prior work like EasyDist [1]), the system can systematically search for valid decomposition strategies across a "decomposition context" for various operators, not just matrix multiplications. This moves the state-of-the-art from a clever, but fixed, optimization pattern to a more general and automated compiler pass. The use of a cost model and an ILP solver (Section 5.5, Page 8) to select among candidate decomposition strategies is also a principled and novel extension.

2. Formalization of Scheduling: While heuristic-based schedulers for communication overlap are common (e.g., gradient bucketing in PyTorch DDP [27] or systems like ByteScheduler [34]), Concerto's formulation of the problem as a formal RCPSP (Section 4, Page 4) is a more principled approach. Although applying RCPSP to scheduling is not new in itself, its application to the specific, fine-grained graph of DL computations and collective communications represents a novel formalization in this domain. It promises more globally optimal solutions than greedy heuristics, within the limits of the model.

---

Weaknesses (Novelty-centric Evaluation)

The novelty of the paper's contributions must be carefully contextualized with respect to existing work.

1. Incremental Novelty in Scheduling Formulation: The core idea of using ILP or established operations research models like RCPSP for scheduling is not fundamentally new. This has been a standard approach in the broader high-performance computing and compiler domains for decades. The novelty is therefore confined to the application of this model to the DL communication problem and the specific encoding used. Furthermore, to make the problem tractable, the authors resort to a heuristic (the "Odd-even Method," Section 4.4.2, Page 5), which moves the solution away from the "near-optimal" promise of the pure ILP formulation. This practical compromise dilutes the novelty of using a formal optimization model.

2. Decomposition Concept is Not New: As the authors acknowledge, the core concept of overlapping communication with dependent computation via decomposition has been established by prior art [47]. Therefore, the claim to novelty rests entirely on the automation and generalization of this concept, not the concept itself. The paper should be very precise about this distinction.

3. Decoupled, Not Joint, Optimization: The paper proposes two main optimization passes: scheduling and auto-decomposition. However, these are performed as separate, sequential steps. The authors themselves acknowledge this limitation in the discussion (Section 9, Page 14), stating that a joint optimization could yield better results. This separation prevents the discovery of more complex trade-offs, for instance, where a slightly sub-optimal decomposition might unlock a vastly superior global schedule. As such, the framework stops short of a truly novel, unified optimization theory for this problem.

---

Questions to Address In Rebuttal

1. Delta vs. Prior Art in Decomposition: The paper’s main claim to novelty over Google's work [47] is generalization. Please provide a concrete example of a non-trivial model architecture (not a standard Transformer) where "auto-decomposition" identifies and executes an effective decomposition strategy that is conceptually different from the einsum/all-reduce pattern and would not be possible with the approach in [47].

2. On the Separation of Concerns: The paper presents scheduling and decomposition as two independent optimization problems. What is the fundamental technical barrier to creating a unified ILP formulation that co-optimizes the choice of decomposition strategy (including the degree 'N' and axis) along with the scheduling of all operators in the graph? Could such a formulation be made tractable, perhaps with new heuristics?

3. Fidelity of the RCPSP Model: The RCPSP model abstracts hardware resources into just two categories: "computation" and "communication" (Section 4.1, Page 4). This ignores critical factors like intra-node vs. inter-node bandwidth differences, network topology, or contention for memory bandwidth. How confident are the authors that schedules produced by this simplified model are truly "near-optimal" on real-world hardware, especially when compared to topology-aware systems like MSCCLang [8]? Is the novelty in formalization outweighed by the lack of model fidelity?

Paper: Cooperative Graceful Degradation In Containerized Clouds Reviewer: The Guardian

---

Summary

The paper proposes Phoenix, a system for cooperative graceful degradation in containerized cloud environments. The core mechanism, termed "diagonal scaling," involves turning off non-critical microservices (containers) during resource-constrained scenarios based on developer-provided "Criticality Tags." The system is composed of a Planner, which generates a prioritized list of containers to run based on application requirements and operator objectives (e.g., fairness, revenue), and a Scheduler, which enacts this plan. The authors formalize the problem as a Linear Program (LP) but implement a heuristic-based algorithm for scalability. The evaluation is conducted on a small-scale real-world Kubernetes cluster (CloudLab) and through a large-scale simulation platform, AdaptLab, which the authors also developed. The results suggest that Phoenix can improve critical service availability and meet operator objectives better than non-cooperative baselines.

Strengths

1. Well-Motivated Problem: The paper correctly identifies a significant gap in current cloud resilience strategies: the disconnect between application-level awareness and infrastructure-level control, particularly in public clouds. The vision for a cooperative framework is compelling.
2. Theoretical Grounding: The formulation of the degradation problem as a Linear Program (Section 4, page 6) provides a clear, formal basis for the system's objectives and constraints. This serves as a valuable, albeit aspirational, gold standard for the problem.
3. Actionable Abstraction: The proposal to use containers as the unit of degradation is a sensible and practical choice. It offers a better granularity than whole VMs without requiring deep, intrusive application modifications, as seen in systems like Defcon [37].
4. Artifact Availability: The authors have made both their system (Phoenix) and their benchmarking platform (AdaptLab) open-source, which is commendable and facilitates reproducibility and follow-on work.

Weaknesses

My primary concerns with this paper center on the foundational assumptions, the generalizability of the evaluation, and the understatement of practical limitations.

1. The Foundational Premise of "Criticality Tags" is Fragile and Unvalidated: The entire security and effectiveness of Phoenix hinges on the assumption that Criticality Tags are provided correctly, honestly, and are static. This assumption is untenable in a real multi-tenant public cloud.

   • Adversarial Behavior: The paper briefly acknowledges "Adversarial or Incorrect Criticality Tags" (Section 7, page 13) but dismisses the concern by suggesting operators can "employ policies such as resource fairness to limit the impact." This is insufficient. What prevents a tenant from tagging all their containers as criticality C1 to monopolize resources during a crunch? A fairness policy might cap their total resources, but Phoenix's logic would still prioritize their (falsely-critical) containers over another tenant's genuinely critical ones up to that cap. This fundamental incentive problem is not addressed.
   • Complexity of Tagging: The paper suggests rule-based and frequency-based methods for tagging (Section 3.2, page 5), but this simplifies a deeply complex issue. The criticality of a microservice can be dynamic and context-dependent (e.g., a reporting service is low-criticality during normal operation but high-criticality during an end-of-quarter rush). The proposed static tagging mechanism is too simplistic for real-world application dynamics.

2. Evaluation Lacks Generalizability and Relies on Modified Benchmarks: The experimental validation does not sufficiently support the broad claims made.

   • Benchmark Modification: The authors state in Section 5 (page 9) that the HotelReservation (HR) application "lacks robust error-handling mechanisms" and is "not entirely crash-proof." They proceed to "implement error-handling logic to prevent request crashes." This is a significant methodological flaw. Instead of evaluating their system on a standard, off-the-shelf benchmark, they have modified the benchmark to be compatible with their degradation strategy. This calls into question whether Phoenix works on typical microservice applications or only on applications that have been specifically hardened to tolerate the abrupt disappearance of their dependencies. This erodes the core claim of a broadly applicable, non-intrusive solution.
   • Over-reliance on Simulation for Scale: The key performance claims at scale (100,000 nodes) are derived entirely from the AdaptLab simulator (Section 6.2, page 11). While the use of real-world dependency graphs is a good starting point, the resource modeling is based on proxies ("calls-per-minute" or "sampled from a long-tailed distribution"). This abstraction ignores complex real-world dynamics like network congestion, cascading failures from resource contention (not just dependency links), and the highly variable time costs of container deletion, migration, and startup. The claim that Phoenix "can handle failures in a cluster of 100,000 nodes within 10 seconds" (Abstract) is based on the planning time in a simulation, not an end-to-end recovery time in a real system of that scale, which is misleading.

3. The Severe Limitation to Stateless Workloads is Understated: The paper confines its scope to stateless workloads, acknowledging this in several places (e.g., Section 1, page 2). However, it justifies this by citing that such workloads comprise "over 60% of resource utilization" [1]. This metric is misleading. Many, if not most, high-value, user-facing applications are stateful. Degrading a stateless frontend is meaningless if the stateful database or caching tier it depends on is terminated. The paper offers no path forward for stateful services, which makes Phoenix a niche solution for a subset of the problem, rather than the general framework it is presented as.

4. Gap Between Optimal LP and Implemented Heuristic: The paper presents an LP formulation but implements a greedy heuristic (Algorithm 1) for scalability. There is no analysis of the heuristic's performance relative to the optimal LP solution. On the small-scale experiments where the LP is tractable (Figure 5), it is unclear if the results for "LPFair" and "LPCost" represent the true optimal solution or Phoenix's heuristic aimed at that objective. If it's the latter, then a crucial comparison is missing: how much revenue or fairness is lost by using a scalable but suboptimal greedy algorithm?

Questions to Address In Rebuttal

1. Please describe a concrete, enforceable mechanism within Phoenix to prevent a tenant in a public cloud from gaming the system by assigning the highest criticality tag (C1) to all of their non-critical containers. How can the operator trust these tags without application-level introspection?

2. The HotelReservation application required modifications to its error-handling to work with Phoenix's degradation. Does this imply that for Phoenix to be effective, applications must already be architected to be resilient to the sudden failure of their downstream dependencies? If so, how does this differ from standard application-level resilience patterns (e.g., circuit breakers), and how does it uphold the claim of being a system that works with general, containerized applications?

3. The performance claims at 100,000 nodes are based on the AdaptLab simulation. Can you provide evidence or a stronger argument for why the simplified resource models used are a sufficiently realistic proxy for a real, production environment of that scale, especially concerning unpredictable overheads like container startup time and network state reconfiguration?

4. For the experiments on CloudLab, where the problem size appears tractable, please provide a quantitative comparison of the solution quality (in terms of fairness and revenue) generated by your heuristic (Algorithm 1) versus the optimal solution generated by the LP solver. What is the optimality gap of your heuristic?

5. Given that stateful services are often the most critical components of an application, could you elaborate on why a solution that exclusively targets stateless workloads is a sufficient first step? What are the fundamental challenges that prevent diagonal scaling from being applied to a stateful container (e.g., a database replica)?

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper presents Phoenix, a framework for cooperative graceful degradation in multi-tenant, containerized clouds. The core idea is to bridge the gap between application-agnostic infrastructure and application-aware resilience. The authors introduce "diagonal scaling"—the targeted deactivation of non-critical microservices during capacity-constrained scenarios—as the primary degradation mechanism. The cooperation is mediated by a simple and practical interface: "Criticality Tags" that application developers assign to their containers.

The Phoenix system comprises a planner that generates a globally-ordered list of microservices to activate based on both application-level tags and operator-level objectives (e.g., fairness, revenue maximization), and a scheduler that enacts this plan on a cluster manager like Kubernetes. The authors evaluate Phoenix through both a real-world deployment on a 200-CPU CloudLab cluster and large-scale simulations using their open-source benchmarking tool, AdaptLab, with traces from Alibaba's production environment. The results demonstrate that this cooperative approach can significantly improve the availability of critical services during large-scale failures compared to non-cooperative baselines.

Strengths

This is a well-written and timely paper that makes a compelling case for a new point in the design space of cloud resilience. Its primary strengths are:

1. Novel and Significant Conceptual Contribution: The paper expertly identifies and addresses a critical gap in cloud resilience management. Current approaches in public clouds treat applications as black boxes, limiting the effectiveness of mitigation strategies. Conversely, highly effective cooperative strategies like Meta's Defcon [37] require deep application integration and are ill-suited for the public cloud model. This work proposes a "gray-box" middle ground that is both powerful and practical. The vision of enabling in-place recovery for partial data center failures, avoiding costly inter-region failovers, is highly impactful.

2. Pragmatic and Elegant Interface: The choice of "Criticality Tags" as the interface between the application and the operator is a standout feature. It is a simple, expressive, and low-friction mechanism that leverages existing tagging capabilities in modern cluster schedulers (Section 3, page 4). This pragmatism dramatically lowers the barrier to adoption for application developers, which is a crucial consideration for any technique intended for wide-scale public cloud deployment.

3. Comprehensive System Design and Evaluation: The authors have not only proposed an idea but also instantiated it in a well-designed system, Phoenix. The evaluation is thorough and convincing. The combination of a real-world deployment on CloudLab with two distinct microservice applications (Section 6.1, page 9) demonstrates feasibility, while the large-scale simulation framework, AdaptLab, provides strong evidence of scalability and performance under realistic conditions (Section 6.2, page 11). The ability of the planner to generate plans for 100,000-node clusters in under 10 seconds is particularly impressive.

4. Contribution to the Field's Vocabulary: The introduction of the term "diagonal scaling" is a useful and intuitive addition to the lexicon of cloud resource management, clearly distinguishing this action from the well-understood horizontal and vertical scaling paradigms. This helps to frame the contribution clearly and provides a useful handle for future work in this area.

Weaknesses

While the paper is strong, its focus and framing give rise to a few weaknesses that temper its immediate, universal applicability.

1. Limitation to Stateless Workloads: The paper's explicit focus on stateless workloads (acknowledged in Section 1, page 2 and Section 7, page 13) is a major limitation. A large number of high-value, critical cloud applications involve stateful components (databases, caches, message queues). Simply terminating and restarting these components is not a viable strategy. While scoping is necessary, the paper would be stronger if it discussed the fundamental challenges of extending this model to stateful services more deeply, as this is where the hardest problems in resilience lie.

2. The "Oracle" of Criticality Tagging: The entire framework's efficacy rests on the assumption that developers can and will provide correct, stable, and honest criticality tags. The paper briefly touches upon automated tagging and adversarial scenarios (Section 7, page 13), but this socio-technical aspect is understated. In a competitive, multi-tenant environment, the incentive to "game the system" by marking all services as maximally critical is high. Operator-level policies like fairness can mitigate this, but the binary nature of diagonal scaling (a service is either on or off) makes it a blunt instrument for enforcing nuanced sharing policies. The problem is not just adversarial behavior but also sheer complexity; determining the true "criticality" of a microservice in a graph of thousands of services is a profound challenge in itself.

3. Interaction with Existing Control Loops: The paper presents Phoenix as a new control loop for resilience, but it does not discuss how it interacts with other, pre-existing control loops common in cloud environments, most notably horizontal autoscaling. For example, if Phoenix deactivates a low-priority service to save capacity, what prevents a Horizontal Pod Autoscaler (HPA) from observing the lack of replicas and immediately trying to scale the service back up? This could lead to resource contention or control loop instability. A robust, production-ready system must be able to coordinate or preempt these other controllers.

Questions to Address In Rebuttal

1. Regarding the focus on stateless workloads: While a full solution for stateful services is future work, could you elaborate on the specific fundamental challenges? For instance, does the core planner/scheduler design need to change to incorporate concepts like data replication costs and recovery point objectives, or is the challenge primarily in the execution "agent," which would need to interact with state-aware operators (e.g., for snapshotting, detaching volumes, or coordinating database failovers)?

2. The framework's success relies on accurate criticality tags. In a multi-tenant public cloud, what prevents a "tragedy of the commons" where all tenants mark their applications as maximally critical to monopolize resources during a crunch? How robust are operator-level policies like fairness against this behavior, especially when diagonal scaling is a binary decision (a service is either running or not), unlike resource throttling which can be applied continuously?

3. Could you please discuss the potential for negative interactions between Phoenix's control loop and other standard Kubernetes controllers like the Horizontal Pod Autoscaler (HPA)? For instance, how would Phoenix prevent an HPA from immediately attempting to counteract a diagonal scaling decision, potentially leading to system instability or "thrashing"? Would Phoenix need to temporarily disable other controllers or coordinate with them directly?

Paper: Cooperative Graceful Degradation In Containerized Clouds Reviewer: The Innovator (Novelty Specialist)

---

Summary

This paper proposes a framework for cooperative graceful degradation between applications and the cloud operator in a public cloud setting. The central claim to novelty rests on bridging the information gap that typically forces operators to treat applications as complete black boxes. The authors introduce "Criticality Tags" on containers as a simple interface for applications to express the relative importance of their microservices. They term the resulting action of selectively deactivating non-critical containers "diagonal scaling." This information is consumed by a new resilience management system, Phoenix, which performs globally-aware planning and scheduling during resource-crunch scenarios to maximize application availability and meet operator objectives like fairness or revenue.

My analysis concludes that while the fundamental action of deactivating less-important components is not new, the paper's primary novel contribution is the specific coordination mechanism for applying this principle in a multi-tenant, containerized public cloud, moving the state of the art from inferential black-box systems to a practical, explicit "gray-box" model.

---

Strengths

The primary strength of this paper, from a novelty perspective, is its successful identification and articulation of a meaningful gap in the existing design space of cloud resilience. The authors correctly position their work between two extremes:

1. Private Cloud / White-Box Systems: The paper references Meta's Defcon [37] (page 2), a system that requires deep, white-box integration and application code modification. The novel "delta" here is the proposal of a mechanism suitable for public clouds, where such modifications are infeasible. Using standard container tagging is a much lower barrier to entry and represents a significant step towards practicality in a multi-tenant environment.

2. Public Cloud / Black-Box Systems: The paper contrasts its approach with prior work [21, 23] (page 2) that relies on inferring application component criticality from infrastructure-level signals. The core conceptual leap forward is moving from error-prone inference to explicit, application-provided signals. This shift from an inferential to a declarative model for inter-layer cooperation is the paper's most significant novel idea.

The introduction of the term "diagonal scaling" (Section 3, page 4) is also a strength. While the concept it describes is related to prior ideas, coining this precise terminology for the act of reducing the number of active microservice types—as opposed to the number of instances (horizontal) or resource allocation per instance (vertical)—is a useful and novel contribution to the field's lexicon.

---

Weaknesses

My main criticism is that the paper sometimes overstates the novelty of the action of graceful degradation itself, when its true innovation lies in the coordination architecture.

1. Conceptual Overlap with "Brownout": The idea of "turning off non-critical containers" is functionally and philosophically very similar to the well-established concept of "brownout" computing [33, 71], where optional application features are "dimmed" or deactivated to conserve resources. Diagonal scaling can be viewed as a specific, container-level implementation of the brownout principle. The paper would be stronger if it explicitly acknowledged this lineage and framed its novelty more precisely as a new, scalable mechanism for achieving brownout in containerized architectures, rather than presenting the idea of deactivation as entirely new.

2. Incremental Novelty of the Interface: The use of tags to signal priority is not, in itself, a revolutionary concept. Kubernetes, for instance, has a "Pod Priority" concept [87] (page 3) that allows for preemption of lower-priority pods. The novelty of "Criticality Tags" is subtle: it is used not just for preemption but as input to a global planner that respects intra-application dependencies and optimizes for cross-application operator objectives. This distinction is crucial but could be made more explicit to better highlight the novelty over existing priority mechanisms.

---

Questions to Address In Rebuttal

To strengthen the paper's claims of novelty, I would expect the authors to address the following points in their rebuttal:

1. Please clarify the fundamental conceptual difference between "diagonal scaling" and the principle of "brownout" [33, 71]. Is the novelty primarily in the implementation (at the container/microservice level) and the cooperative control plane, rather than in the core idea of deactivating non-essential functionality? A more direct comparison would help situate the contribution relative to this important prior art.

2. The paper’s core mechanism is an explicit interface (Criticality Tags) that supersedes the inference-based techniques of systems like Narya [21]. However, a key challenge in public clouds is adoption. How does your proposed architecture's novelty hold up in a more realistic "partially-adopted" scenario where the operator must manage a mix of explicitly tagged applications and legacy, untagged, black-box applications? Does Phoenix revert to inference for the latter, and if so, how are decisions arbitrated between the two classes of applications?

3. Could you provide a more detailed comparison between the semantics and expressive power of your "Criticality Tags" and Kubernetes' native Pod Priority and Preemption mechanism [87]? Specifically, Pod Priority is an integer-based system used primarily for scheduling and eviction decisions. How is your multi-level C1, C2,... scheme fundamentally more expressive or better suited for the global optimization task performed by the Phoenix planner?

Reviewer: The Guardian

---

Summary

This paper presents Copper and Wire, a new service mesh architecture designed to improve policy expressiveness and performance. The authors introduce Abstract Communication Types (ACTs) to decouple policies from specific dataplane implementations, a new policy language (Copper) that uses "run-time contexts" to specify policies over request sequences, and a control plane (Wire) that uses a MaxSAT formulation to optimize the placement of sidecars. A key component of the proposed system is a novel eBPF-based mechanism for propagating these run-time contexts without modifying application code. The evaluation, conducted on three microservice benchmarks, claims significant reductions in policy code complexity (up to 6.75x fewer lines), tail latency (up to 2.6x smaller), and resource consumption (up to 39% fewer CPU resources) compared to standard Istio deployments.

While the proposed abstractions are intriguing and the performance gains appear notable, a rigorous examination reveals several methodological weaknesses, questionable assumptions about the dataplane, and claims that may not hold under real-world conditions. The core contribution of transparent context propagation, in particular, seems to rely on a non-transparent protocol modification, and the evaluation may be based on an oversimplified cost model.

Strengths

1. Well-Motivated Problem: The paper correctly identifies significant, widely acknowledged pain points in current service mesh frameworks: poor policy expressiveness for sequential operations, high resource overhead, and challenges with dataplane heterogeneity.
2. Principled Optimization Approach: The use of a MaxSAT solver (Section 5, p.8) to determine sidecar placement is a formal, principled approach to the resource optimization problem, moving beyond the naive "deploy everywhere" default of many current systems.
3. Decoupling Abstractions: The concept of Abstract Communication Types (ACTs) (Section 4.1.1, p.5) is a sound architectural principle for addressing dataplane heterogeneity. It provides a clear path for integrating new proxies without requiring changes to the central control plane logic.

Weaknesses

1. The Context Propagation Mechanism is Fundamentally Non-Transparent: The paper’s claim to transparency is questionable. The eBPF add-on (Section 6, p.9) works by "add[ing] the raw bytes of the context in outgoing requests as a new CTX HTTP/2 frame." This is a direct modification of the L7 protocol. It breaks the contract of a truly transparent sidecar, which should interoperate with any compliant client/server. This approach will fail for any service that does not expect or cannot parse this custom frame. Furthermore, the paper completely omits any discussion of how this mechanism functions in the presence of inter-service TLS encryption. If traffic is encrypted, the eBPF hook cannot inspect or inject headers/frames without TLS termination, which would re-introduce significant overhead and complexity at every hop, undermining the entire performance premise.

2. Oversimplified and Potentially Unrealistic Cost Model: The Wire optimizer's MaxSAT formulation relies on a static, user-provided cost c for each sidecar type (Section 5, p.8). This is a gross simplification. In reality, the overhead (cost) of a sidecar is not a static value; it is a complex function of the specific policies it enforces, the request rate, and the request payload size. A SetHeader operation is not free, yet the "free-policy" classification (Section 5, p.8) allows the optimizer to treat it as such for placement purposes, which could artificially inflate the perceived benefits of the Wire optimizer.

3. Scalability Concerns for Dynamic Environments: The evaluation reports that the MaxSAT solver can take up to 9.8 seconds to find an optimal placement for the largest production graphs (Section 7.2.3, p.13). While this may be acceptable for initial deployment, it raises serious concerns about the system's agility. In a dynamic cloud-native environment with frequent deployments, scaling events, and policy updates, does every minor change require a full, multi-second re-solve? The paper does not address the latency of reconfiguration, a critical metric for production control planes.

4. The "Istio++" Baseline is Insufficiently Strong: The authors introduce an "Istio++" baseline to represent an optimized state (Section 7.2.1, p.11). While better than the naive default, it is still a weak adversary. A skilled operator could use existing Istio/Envoy features (e.g., Lua filters or custom WASM extensions) to achieve context propagation without application modification. This would be complex, but it is the true state-of-the-art for such problems. By not comparing against such a configuration, the paper fails to demonstrate superiority over what is currently possible, albeit difficult.

5. Evidence for Expressiveness is Limited to Simple Cases: The paper claims Copper simplifies writing "complex policies" (Abstract, p.1), but the examples provided (P1-P4 in Table 3, p.10) are primarily simple header manipulations, routing, and access control based on request paths. It is not demonstrated how Copper would handle truly complex, stateful policies, such as conditional request throttling based on a prior authentication flow's outcome, or dynamic request shadowing based on payload content. The regex-based context matching could become unwieldy and unmaintainable for such scenarios.

Questions to Address In Rebuttal

1. Regarding context propagation (Section 6, p.9): Please clarify how the eBPF mechanism handles encrypted (TLS) traffic between services. Does it require service mesh-level TLS termination at every hop where context might be needed, and if so, have you measured the performance impact of this requirement? Furthermore, please justify how introducing a custom, non-standard HTTP/2 frame constitutes a "transparent" solution.

2. Regarding the optimization model (Section 5, p.8): The MaxSAT solver takes up to 9.8s on large graphs. In a dynamic environment, what is the expected reconfiguration latency when a single policy is updated or a single service is scaled? Does this trigger a full re-solve of the entire graph?

3. Regarding the "free-policy" definition (Section 5, p.8): Please justify the classification of policies that perform actions like SetHeader as "free." While they may not require cross-request state, they are not zero-cost in terms of CPU. Could this classification lead to suboptimal placements in scenarios where a "free" policy is placed on a hot-path service, creating a bottleneck?

4. Regarding dataplane heterogeneity: The paper's core eBPF mechanism appears tightly coupled to HTTP/2. How would the Copper/Wire system propagate context for other L7 protocols commonly found in microservice environments, such as Thrift, Kafka, or raw TCP streams, without requiring a separate, custom-built eBPF parser for each?

Paper Title: Copper and Wire: Bridging Expressiveness and Performance for Service Mesh Policies Reviewer Persona: The Synthesizer (Contextual Analyst)

---

Summary

This paper presents Copper and Wire, a novel, co-designed service mesh architecture aimed at solving two of the most pressing problems in the field: the difficulty of expressing complex, cross-service communication policies and the significant performance overhead imposed by current mesh implementations. The core contribution is a holistic rethinking of the abstractions used for mesh policy. The authors introduce Abstract Communication Types (ACTs) to decouple policies from specific dataplane implementations, and more importantly, they elevate the "run-time context"—the causal sequence of requests—to a first-class citizen in their policy language, Copper. This allows developers to write intuitive policies over entire request chains. This high-level specification is then consumed by Wire, a performance-oriented control plane that leverages policy semantics and the application graph to generate an optimal, minimal deployment of sidecar proxies. The system is enabled by a lightweight eBPF add-on for efficiently propagating context without requiring a sidecar at every service.

In essence, the work recasts service mesh policy from a per-service, endpoint-centric configuration problem into a holistic, application-aware optimization problem, akin to compiling a high-level program down to efficient machine code.

Strengths

1. A Powerful Central Abstraction: The most significant contribution of this work is the conceptual leap of making the request "context" a primary primitive for policy specification (Section 4.1.2, page 5). This directly maps to the mental model developers have of their applications, where a user action triggers a cascade of internal service calls. By allowing policies to be written as regular expressions over these service chains (e.g., "frontend.*catalog"), Copper elegantly sidesteps the brittleness and complexity of today's approaches, where developers must manually stitch together multiple per-service policies and even modify application code to propagate context (as illustrated beautifully in Figure 1). This is a fundamental shift that connects the service mesh policy layer to the well-established domain of distributed tracing, using trace context not just for observability but for active policy enforcement.

2. Elegant Co-design of Language and System: The paper's strength lies in its holistic design. This is not merely a new DSL, but a complete system where each component complements the others. The semantics of the Copper language (e.g., [Egress] annotations on actions, as described in Section 4.1.3) provide crucial information that the Wire control plane's optimizer directly uses in its MaxSAT formulation (Section 5, page 8). This tight coupling between the high-level language and the low-level optimizer is what enables the impressive performance gains. It's a classic example of how raising the level of abstraction can unlock new optimization opportunities that are impossible when working with low-level, imperative configurations.

3. Addressing Dataplane Heterogeneity: The paper correctly identifies the tight coupling between control planes and dataplanes as a major limitation in the current ecosystem. The introduction of Abstract Communication Types (ACTs) and dataplane-provided interfaces is a thoughtful solution. It provides a principled path towards a truly "pluggable" dataplane, allowing operators to mix-and-match proxies (e.g., a feature-rich Envoy with a lightweight Cilium-proxy) based on the specific policy requirements of different services. This is a practical and important contribution that could have a significant impact on the evolution of the service mesh landscape.

4. Strong and Convincing Evaluation: The evaluation in Section 7 is comprehensive and effectively supports the paper's claims. By comparing against both standard Istio and an improved Istio++ baseline, the authors demonstrate that their performance gains are not just from avoiding naive deployments but from fundamentally better optimization. The results showing up to 6.75x fewer lines of policy code (Table 3), 2.6x lower tail latency, and 39% lower CPU usage (Figures 9 and 10) are substantial. The inclusion of an evaluation on real-world production traces from Alibaba (Section 7.2.2) further grounds the work in reality, showing its potential effectiveness on large, complex application graphs.

Weaknesses

While the technical vision is compelling, its path to real-world impact faces some challenges that could be discussed further.

1. The "Clean Slate" Adoption Barrier: The work proposes a ground-up redesign, which, while technically elegant, presents a significant adoption hurdle. The ecosystem is heavily invested in existing APIs like Istio's. The paper touches on migration in Section 8, suggesting dataplane vendors would need to provide Copper interfaces and compilers. This is a very high bar. The work would be even more impactful if it explored a more gradual migration path. Could the Copper abstractions be used to generate configurations for existing control planes like Istio as an intermediate step, providing the expressiveness benefits while the optimization framework is adopted later?

2. Scope of Context Propagation: The current eBPF-based context propagation mechanism (Section 6, page 9) is cleverly designed for synchronous, RPC-style communication (like gRPC over HTTP/2). However, modern microservice architectures are increasingly reliant on asynchronous communication via message queues and event buses (e.g., Kafka, RabbitMQ). It is unclear how the notion of a causal "run-time context" would be defined and propagated across these asynchronous boundaries. This is not a flaw in the current work but a significant question about the generalizability of the proposed mechanism to a broader class of distributed applications.

3. Implicit Handling of Policy Conflicts: The paper proposes a very expressive policy language, which naturally raises the question of how to handle conflicting policies. For instance, what happens if one policy applies a RouteToVersion action and another applies a Deny action to the same request? The authors acknowledge this as an interesting future direction in their conclusion (Section 8). While understandable to scope out, the lack of a defined semantics for policy composition or conflict resolution is a notable omission for a system intended for production use. The richness of the language makes this problem more acute than in simpler systems.

Questions to Address In Rebuttal

1. Regarding the adoption challenge: Could the authors elaborate on a potential incremental adoption path? For instance, could the Copper/Wire architecture coexist with an existing Istio control plane, perhaps managing a subset of services, to allow organizations to migrate gracefully rather than requiring a "rip and replace" approach?

2. Regarding the scope of context: How do the authors envision extending the concept of "run-time context" and the corresponding eBPF propagation mechanism to applications that use asynchronous communication patterns, such as event buses or message queues, which do not have a direct request-response structure?

3. Regarding policy conflicts: The paper acknowledges this as future work. However, could the authors comment on whether the proposed abstractions (ACTs, contexts) might themselves offer a more principled way to detect or even resolve such conflicts, perhaps by defining policy priorities or composition rules as part of the language itself?

Reviewer: The Innovator (Novelty Specialist)

Summary

This paper presents Copper and Wire, a co-designed service mesh architecture aimed at improving policy expressiveness while minimizing performance overhead. The authors identify three core novel contributions: 1. Abstract Communication Types (ACTs): A new abstraction layer for network communication primitives (requests, responses, connections) and their associated actions, designed to decouple policy logic from specific dataplane implementations. 2. Context-based Policies (Copper): A new policy language, Copper, that operates on these ACTs. Its central feature is the ability to define policies over "run-time contexts," which represent the causal sequence of service interactions leading to a communication event. These contexts are expressed as regular expressions over service names. 3. Optimized Control Plane (Wire): A new control plane, Wire, that leverages semantic information from the ACT interfaces (e.g., [Ingress]/[Egress] annotations) and the application's communication graph to formulate a sidecar placement problem as a MaxSAT instance. This allows it to deploy a minimal set of sidecars to enforce policies correctly.

The core novelty claim is not any single one of these components in isolation, but rather their synergistic integration, creating a "semantic bridge" from high-level policy expression down to low-level, optimized resource deployment.

Strengths (in terms of novelty)

1. The Semantic Bridge between Policy and Placement: The most significant novel contribution is the co-design of the policy language and the control plane optimizer. Current control planes like Istio treat dataplane configurations as an opaque target. In contrast, Wire uses the semantic annotations ([Ingress], [Egress], "free-policy" classification) derived from the ACT interfaces (Section 4.1.3, page 6) to reason about where a policy action can be validly enforced. This allows for a principled optimization that is not possible in existing systems. This tight coupling between the language's semantics and the control plane's optimization logic is genuinely new.

2. Novel Policy Abstraction and Representation: While the desire for more expressive policies is not new, the specific abstractions proposed are. The concept of a context pattern expressed as a regular expression (Section 4.2, page 7) is an elegant and novel way to specify policies over complex request sequences without requiring developers to write separate rules for each intermediate service. This is a distinct and arguably more flexible representation than the explicit tree structures proposed in prior work.

3. Pragmatic and Novel Implementation of Context Tracking: The system's viability rests on low-overhead context tracking. The choice to use an eBPF add-on is not novel in itself (Cilium is built on eBPF). However, the specific implementation detailed in Section 6 (page 9) is novel. The technique of adding the context as a raw custom HTTP/2 frame to avoid complex L7 header parsing within the constraints of eBPF is a clever piece of engineering that makes the high-level concept of contexts practical.

Weaknesses (in terms of novelty and differentiation from prior art)

1. Conceptual Overlap with Prior Work on Expressive Policies: The paper positions itself against mainstream service meshes like Istio but does not sufficiently differentiate its core ideas from recent academic work. Specifically, Grewal et al. (HotNets '23) [24] also proposed a system for "Expressive Policies For Microservice Networks" using "service tree" abstractions to capture request flows. The "run-time context" in this paper appears to be a linear/string-based representation of a similar concept. The paper's novelty would be strengthened by a direct and detailed comparison, articulating why the regex-based context is a significant advancement over service trees, beyond syntactical differences.

2. The Goal of Dataplane Heterogeneity is Not New: The paper claims to better support dataplane heterogeneity. ServiceRouter (OSDI '23) [32] was also an attempt to use multiple dataplanes in a single mesh. While the mechanism proposed here (ACTs as a formal interface) is a cleaner and less intrusive approach than ServiceRouter's common RPC library, the foundational goal is not entirely novel. The contribution should be framed more precisely as a novel architecture for achieving heterogeneity, rather than claiming the goal itself is new.

3. Limited Exploration of the Abstraction's Expressiveness: The context is represented as a linear sequence of services. It is not clear if this abstraction is powerful enough to express policies that depend on non-linear or conditional paths (e.g., "apply policy if the request path included service A but not service B"). The novelty of the regex abstraction is tied to its expressiveness, and the limits of this are not fully explored.

Questions to Address In Rebuttal

1. Please provide a detailed comparison of Copper's context patterns with the "service tree" abstractions proposed by Grewal et al. [24]. What classes of policies can be expressed by one and not the other? Is the primary contribution a more ergonomic syntax, or is there a fundamental difference in expressive power?

2. The regex-based context appears to capture a linear path of execution. How would the Copper language express policies that depend on more complex path properties, such as branching (e.g., a request from A that goes to either B or C, but the policy at D depends on which one was chosen) or negative constraints (e.g., a path that did not include a specific service)?

3. The ACT abstraction relies on dataplane vendors to provide implementations that are semantically equivalent (e.g., SetHeader should behave identically across dataplanes). How does the framework handle subtle but important semantic differences in the implementation of actions across different proxies? For example, what if RouteToVersion in Dataplane A has different retry logic or failure semantics than the same-named action in Dataplane B? Does the abstraction leak?

4. The core optimization problem is framed around minimizing the number of sidecars. With the emergence of architectures like "ambient mesh" that move the proxy function to a shared node-level agent, how does the novelty of Wire's placement optimization hold up? Is the contribution fundamentally tied to the per-pod sidecar model, or can the semantic-driven optimization be adapted to these newer models?

Title: CRUSH: A Credit-Based Approach for Functional Unit Sharing in Dynamically Scheduled HLS Reviewer: The Guardian

---

Summary

The authors present CRUSH, a resource sharing methodology for dynamically scheduled High-Level Synthesis (HLS). The paper identifies two primary sources of deadlock introduced by naive resource sharing: head-of-line blocking at the shared resource's output and dependency violations from fixed-order arbitration at the input. To address these, CRUSH proposes a credit-based flow control mechanism to prevent output blocking and a priority-based arbiter to resolve input contention without creating cyclic dependencies. The methodology is complemented by a set of heuristics to determine which operations to group for sharing and their access priorities, with the goal of preserving the circuit's Initiation Interval (II). The evaluation, performed using the Dynamatic HLS compiler, compares CRUSH against a baseline without sharing and a prior total-order-based sharing approach [33], demonstrating reductions in resource utilization and, most notably, in optimization runtime.

Strengths

1. Clear Problem Formulation: The paper does an excellent job of dissecting the sharing problem in dataflow circuits. The examples in Figure 1 (Page 3) clearly and correctly illustrate the mechanisms of both head-of-line blocking deadlock and fixed-order arbitration deadlock, providing a solid foundation for the proposed solutions.

2. Sound Core Mechanisms: The two primary mechanisms proposed appear technically sound for the specific problems they target. The credit-based scheme (Section 4.1) is a well-established technique for preventing buffer overflow and backpressure-induced deadlock, and its application here to break the cycle with successor components is logical. Similarly, the use of priority-based arbitration (Section 4.2) over a fixed round-robin scheme is a standard and correct way to avoid deadlocks when processing inputs with inter-dependencies.

3. Demonstrated Generality: The authors strengthen their claims of applicability by evaluating CRUSH not only on the baseline BB-centric HLS flow from [29] but also on a more recent "fast token" flow from [21] which omits BB structure (Section 6.5, Page 11-12). The consistent resource savings reported in Table 3 provide compelling evidence that the CRUSH wrapper and its control logic are indeed agnostic to the higher-level organization of the dataflow graph.

Weaknesses

My primary concerns with this work lie in the rigor of its claims, the framing of its contribution relative to prior work, and a mismatch between its motivation and evaluation.

1. Insufficient Justification for Credit Sizing: The paper's rule for credit allocation, NCC,op = Φop + 1 (Equation 3, Page 8), is a cornerstone of the implementation, yet its sufficiency is justified by an informal, heuristic argument rather than a rigorous proof. The claim that "at most one token staying in the output buffer" in a steady state seems plausible but is not formally substantiated. The argument hinges on the relationship II ≥ |G|, which is enforced by the grouping heuristic (R2). This makes the performance guarantees of the credit mechanism dependent on a heuristic, which is not ideal. A formal analysis is required to prove that this sizing is always sufficient to prevent credit starvation from becoming a performance bottleneck.

2. Uncharitable Comparison to Prior Work: The central argument for CRUSH's performance advantage over the prior "In-order" method [33] relies on the example in Figure 2 (Page 4). The figure portrays the total-order-based approach as being forced into a pessimistic schedule with II=4. However, it is not established that this is a fundamental limitation of the total-order paradigm. It is conceivable that a more intelligent scheduling algorithm within the total-order framework could find a better static ordering that preserves the II. The paper presents this prior work as a strawman, attacking what appears to be a naive implementation of it rather than addressing its theoretical peak performance.

3. Mismatch Between Motivation and Evaluation Suite: The abstract and introduction motivate the need for dynamically scheduled circuits by highlighting their ability to handle "irregular control flow or unpredictable memory accesses." However, the evaluation in Section 6 is conducted almost exclusively on kernels from the PolyBench suite. These benchmarks are characterized by regular, statically-analyzable loop structures and predictable memory access patterns—the very domain where statically scheduled HLS excels. They are not the workloads that would stress-test the dynamic capabilities CRUSH claims to preserve. While gsum and gsumif are better examples, the overwhelming reliance on PolyBench weakens the validation of the core motivation.

4. Overstated Claims of Sharing Effectiveness: The results in Table 2 (Page 10) show that for 9 out of the 11 benchmarks, CRUSH achieves the exact same level of functional unit sharing (e.g., "1 fadd 1 fmul") as the In-order method. The ability of CRUSH to find more sharing opportunities is only demonstrated on gsum and gsumif. The most significant and consistent advantage shown is the 90% reduction in optimization time. Therefore, the primary contribution appears to be a much faster heuristic for resource sharing, not necessarily a universally more effective one in terms of raw sharing potential. The abstract's claim of achieving an "average reduction of 12% DSPs" over the prior strategy is misleading, as this average is skewed by two specific benchmarks and is not representative of the general case shown in the results.

Questions to Address In Rebuttal

The authors should address the following points in their rebuttal:

1. Regarding the comparison to the total-order-based approach [33]: Can you elaborate on why the II=4 scenario in Figure 2a is a fundamental limitation of that approach, rather than a limitation of a specific, simplistic ordering choice? Could a more advanced scheduler operating within the total-order constraint achieve the optimal II=2?

2. Regarding credit allocation: Can you provide a more formal proof or a worst-case analysis demonstrating that NCC,op = Φop + 1 credits are always sufficient to maintain the target II and that credit starvation will not occur under any condition allowed by your grouping heuristics?

3. Regarding the evaluation suite: Please justify the choice of the highly regular PolyBench suite for evaluating a technique intended for dynamically scheduled circuits. How can we be confident that the conclusions drawn from these benchmarks generalize to the irregular applications that motivate the work?

4. Regarding the cost function (Equation 2, Page 5): This function is critical for guiding the grouping heuristic (Algorithm 1). How were the costs for the wrapper components (Cwp) and shared units (CT) determined? How sensitive are the final sharing decisions and resulting circuit quality to these cost parameters?

Reviewer: The Synthesizer (Contextual Analyst)

---

Summary

This paper addresses the critical problem of functional unit sharing within the context of dynamically scheduled High-Level Synthesis (HLS). While resource sharing is a standard optimization in traditional static HLS, it introduces unique challenges in dynamic, dataflow circuits, namely the potential for performance degradation and correctness issues like deadlock. The authors identify the limitations of prior work, which relies on a restrictive total-token-order, forcing in-order access to shared resources and thus sacrificing performance opportunities.

The core contribution is CRUSH, a novel sharing mechanism that elegantly solves this problem by borrowing a well-established concept from the field of interconnection networks: credit-based flow control. By creating a localized sharing wrapper that manages access via credits tied to output buffer availability, CRUSH provably avoids deadlock from head-of-line blocking. Furthermore, it employs a priority-based arbiter guided by scalable heuristics to permit out-of-order access to the shared unit, preserving the performance benefits of dynamic execution. The authors demonstrate that this approach significantly reduces resource usage (especially scarce DSPs) with negligible performance impact and a substantial reduction in optimization time compared to the state-of-the-art.

Strengths

1. Elegant Cross-Domain Technology Transfer: The paper's primary strength is its insightful application of a mature solution (credit-based flow control) from an adjacent domain (interconnection networks, NoCs) to a challenging problem in HLS. The authors correctly identify that the deadlock problem in sharing (Figure 1b, page 3) is a manifestation of head-of-line blocking, and that credits are the canonical solution. This is an excellent example of conceptual synthesis that moves the field forward by introducing a more robust and flexible mechanism than domain-specific, ad-hoc solutions.

2. Clear Problem Formulation and Motivation: The paper does an outstanding job of motivating its contribution. The simple code example and the accompanying diagrams in Figure 1 and Figure 2 (pages 3 and 4) are exceptionally clear. Figure 1 effectively dissects the deadlock problem into two root causes (head-of-line blocking and fixed-order dependencies), while Figure 2 provides a compelling performance argument against the prior total-order approach. This clarity makes the value proposition of CRUSH immediately apparent.

3. Strong Emphasis on Generality and Practicality: The authors demonstrate that CRUSH is not tied to a specific HLS methodology. The evaluation in Section 6.5 (page 11), where CRUSH is successfully applied to a completely different dataflow HLS strategy ("Fast token"), is a powerful testament to the modularity and generality of the proposed mechanism. This suggests that CRUSH could be adopted as a standard "plug-in" sharing solution in various dataflow HLS frameworks. The staggering 90% reduction in optimization runtime further underscores the practical advantage over prior, MILP-heavy techniques.

4. Well-Designed and Scalable Heuristics: Rather than relying on computationally expensive exact methods, the paper proposes simple and scalable heuristics based on Strongly Connected Component (SCC) analysis for grouping units and setting access priorities (Section 5.2 and 5.3, page 7). This is a pragmatic choice that directly addresses the compile-time scalability issues that often plague advanced HLS optimizations.

Weaknesses

1. Under-explored Impact on Timing Closure: The paper acknowledges that the sharing wrapper adds combinational logic (muxes, arbiters) that can increase the critical path and thus degrade the maximum clock frequency (CP) (Section 6.4, page 11). While the results show this impact is manageable for the chosen benchmarks, the paper dismisses the concern by suggesting it can be "mitigated by enhancing Dynamatic's timing model." This feels somewhat incomplete. For designs targeting high frequencies, this wrapper logic could become the limiting factor, and a more in-depth discussion of the timing implications and potential timing-aware heuristics would strengthen the work's practical claims.

2. Limited Scope of Evaluated Shared Resources: The evaluation exclusively focuses on sharing floating-point arithmetic units. While these are an excellent and highly relevant target due to their high cost in terms of DSP blocks, it leaves open the question of how CRUSH would apply to other important resource types. For example, sharing single-port Block RAMs or stateful custom functional units could introduce new challenges not covered by the current model, which implicitly assumes stateless, pipelined units.

3. Potential Optimality Gap of Heuristics: The heuristics presented are sensible and scalable, but their limitations are not deeply explored. The paper rightly avoids complex analysis for the sake of speed, but it would be valuable to understand if there are specific classes of dataflow graphs (e.g., those with many parallel, independent operations contending for one resource) where the SCC-based priority assignment might lead to a sub-optimal schedule that unnecessarily increases the initiation interval (II).

Questions to Address In Rebuttal

1. Regarding the timing impact, can the authors elaborate on the complexity of the arbitration logic as the number of sharing operations (|G|) increases? Does the critical path scale linearly, or is there a more complex relationship? Could the proposed heuristics be made timing-aware, for instance, by prioritizing operations that are already on the circuit's critical path?

2. Could the authors comment on the applicability of CRUSH to stateful or non-pipelined shared resources, such as a memory controller or a functional unit with multi-cycle latency but no internal pipeline? Would the credit-based mechanism still be sufficient, or would it require fundamental extensions to handle resource state and reservation?

3. The core of the access priority heuristic (Algorithm 2, page 8) relies on the topological order of SCCs. In cases where multiple candidate operations reside within the same large SCC, the choice is arbitrary. Have you considered secondary heuristics for such cases to further optimize performance, or do you find in practice that this situation is rare or has minimal performance impact?

Paper Title: CRUSH: A Credit-Based Approach for Functional Unit Sharing in Dynamically Scheduled HLS Review Form: The Innovator (Novelty Specialist)*

---

Summary

The paper addresses the problem of functional unit (FU) sharing in high-level synthesis (HLS) for dynamically scheduled, dataflow circuits. The central challenge is that naive sharing can introduce resource dependencies that lead to circuit deadlock, specifically through head-of-line blocking. The authors propose CRUSH, a sharing architecture that uses a credit-based flow control mechanism to provably avoid such deadlocks. This mechanism decouples the shared resource from its consumers, allowing operations to access the shared FU out-of-order, a capability lacking in the closest prior art. The core idea is to adapt a well-known deadlock avoidance technique from the network-on-chip and interconnect domain to the specific problem of HLS resource sharing.

---

Strengths

1. Novel Application of a Proven Concept: The primary novel contribution is the application of credit-based flow control to the problem of FU sharing in dataflow HLS. While credit-based flow control is a canonical technique for deadlock avoidance in packet-switched networks (e.g., Dally and Towles, "Principles and Practices of Interconnection Networks"), its adaptation to arbitrate access among HLS operations sharing a pipelined functional unit is, to my knowledge, new.

2. Clear Conceptual Delta from Prior Art: The paper clearly articulates its improvement over the most relevant prior work [33]. The work in [33] enforces a total token order (e.g., based on basic block order) to prevent deadlock. This is a restrictive, global scheduling constraint. CRUSH replaces this with a localized, protocol-based solution. This conceptual shift from a global scheduling constraint to a local flow control protocol is the paper's most significant innovative step.

3. Enabling Out-of-Order Access: A direct consequence of this novel approach is the ability to handle out-of-order requests to the shared resource (as illustrated in Figure 2b, page 4). This capability was explicitly prohibited by the total-order approach in [33] and is a key enabler for seizing more sharing opportunities without incurring performance penalties, which is a significant advancement for dynamically scheduled systems.

---

Weaknesses

1. Limited Foundational Novelty: The core mechanism—credit-based flow control—is not new. The authors themselves cite prior work from the interconnect domain ([17], [40]). The contribution is one of elegant adaptation and application, not the invention of a fundamentally new deadlock-avoidance algorithm. The paper's framing should be careful to claim novelty in the context and adaptation rather than the mechanism itself.

2. Heuristics Lack Deep Novelty: The supporting heuristics for grouping FUs and determining access priority (Section 5.2 and 5.3, page 7) are based on standard and well-understood graph analysis techniques (Strongly Connected Components, topological ordering). While practical and effective, these algorithms are not novel contributions in their own right. The novelty of the work rests almost entirely on the architectural wrapper for sharing, not the decision-making process for creating the sharing groups.

3. Simple Credit Allocation Model: The rule for credit allocation (NCC,op = Φop + 1, described in Section 5.4, page 8) is presented as a simple heuristic. While intuitive, it lacks a formal proof of optimality or sufficiency for all cases beyond the steady-state argument provided. A more rigorous analysis would strengthen the claim that this simple rule is sufficient to maintain the target initiation interval (II) without over-provisioning resources.

---

Questions to Address In Rebuttal

1. The core mechanism is adapted from the interconnect domain. Beyond the obvious mapping of FUs to routers and operations to packets, what were the non-trivial, non-obvious adaptations required to make credit-based flow control work effectively and efficiently in the context of HLS-generated dataflow circuits? For instance, how does the handling of pipelined units and loop-carried dependencies in HLS differ from credit management in a typical NoC router?

2. The access priority heuristic (Algorithm 2) defaults to an arbitrary ordering for operations within the same Strongly Connected Component (SCC), stating "any priority between opi, opj is accepted". Given that a poor choice of priority can degrade performance (as shown in Figure 4c and 4f, page 6), this seems like a potential weakness. Is this relaxation a fundamental limitation, or was a more sophisticated intra-SCC prioritization scheme considered and discarded? Does this not re-introduce a need for performance-driven scheduling, which the protocol was meant to avoid?

3. Regarding the credit allocation rule (Section 5.4), could you provide a more formal argument or proof that Φop + 1 credits are both necessary and sufficient to prevent performance degradation (i.e., maintain the II) under all conditions, not just in the specific steady-state scenario described? Is it possible for transient behavior or complex dependencies to require more than one "extra" credit?

Papaer Title: DarwinGame: Playing Tournaments for Tuning Applications in Noisy Cloud Environments

Reviewer Persona: The Guardian

---

Summary

The authors present DarwinGame, a framework for performance tuning of applications specifically within noisy, shared cloud environments. The central thesis is that existing autotuners are fundamentally flawed for this context as they incorrectly assume a stable, interference-free execution environment. DarwinGame proposes a complex, multi-phase tournament structure (Swiss, double elimination, barrage) to systematically pit different parameter configurations against each other via co-located execution. The goal is to identify a configuration that is not only fast but also robust to performance variations caused by interference. The paper claims this heuristic-driven approach significantly outperforms state-of-the-art tuners, achieving execution times closer to a theoretical optimum with much lower performance variability.

---

Strengths

1. Problem Motivation: The paper correctly identifies a significant and increasingly relevant problem. The assumption of a dedicated, noise-free environment for performance tuning is indeed a major limitation of many existing autotuning systems when applied to modern cloud platforms. The motivation presented in Section 1 is sound.

2. Core Comparison Mechanism: The fundamental idea of co-locating multiple application instances with different configurations to assess their relative performance under similar noise conditions is a clever way to bypass the problem of non-stationary, uncontrollable background noise. This is a direct and logical approach to the problem.

3. Empirical Breadth: The authors have conducted an extensive set of experiments across four non-trivial applications (Redis, GROMACS, FFmpeg, LAMMPS), comparing their approach against multiple established baselines (ActiveHarmony, OpenTuner, BLISS) and theoretical bounds (Optimal, Exhaustive Search). The inclusion of an ablation study (Figure 16) demonstrates a commendable effort to understand the contribution of individual design components.

---

Weaknesses

My primary concerns with this work center on its lack of theoretical grounding, the introduction of a new, complex set of untunable parameters, and significant threats to the validity of the experimental conclusions.

1. Arbitrary and Over-Engineered Design: The entire tournament structure appears to be a fragile collection of ad-hoc heuristics. The authors themselves admit the design is "driven by intuitive choices, instead of claiming any theoretical bounds" (Section 3.2, page 5). This is not acceptable for a rigorous systems paper.

   • Why this specific progression of Swiss, followed by double elimination, followed by barrage? There is no principled justification provided for this sequence over any other.
   • The "Consistency Score" (Section 3.4, Figure 7), defined as the average of 1/ranking, is a completely arbitrary metric. It lacks any statistical foundation. Why is this a better measure of robustness than, for example, the coefficient of variation, variance, or an analysis of tail-latency behavior?
   • The paper is littered with magic numbers. The work-done deviation for early termination d is set to 10% (Section 3.2). The search space is divided into n_r=10,000 regions (Section 3.3). The authors provide a cursory sensitivity analysis showing small variations for these values, but this does not prove generality across different applications, search space topologies, or noise characteristics. This framework seems to replace one tuning problem (application parameters) with another, arguably more complex one (tuner hyperparameters).

2. Flawed Experimental Premise (Internal Validity): The core experimental methodology of co-location creates an artificial environment that is not representative of general cloud noise.

   • By co-locating up to 32 instances of the same, often resource-intensive, application on a single VM (Section 4), the authors are not measuring robustness to typical cloud interference (e.g., from a web server, a small database, etc.). Instead, they are measuring which configuration best survives a highly specific, self-induced, high-contention scenario where all contenders are fighting for the exact same resources (cache, memory bandwidth, core schedulers).
   • The winning configuration from DarwinGame may simply be the one that is least sensitive to contention from its own clones, which is not the same as being the most performant or robust configuration in a production environment with a diverse mix of co-located tenants. The evaluation is circular: it validates a method designed for a high-contention tournament by running a high-contention tournament.

3. Unfair Comparison to Baselines: The paper builds a narrative that existing tuners are "fundamentally unaware" of noise. However, the experimental design for these baselines is not sufficiently detailed to ensure a fair comparison.

   • A standard practice when using a tuner like OpenTuner in a noisy environment is to perform multiple measurements for each configuration and average the results to mitigate noise. The paper does not specify if this was done for the baseline tuners. If each configuration was sampled only once for the baselines, while DarwinGame's configurations are inherently "sampled" multiple times across different rounds and opponents, then the comparison is fundamentally inequitable. DarwinGame's superiority may stem from this methodological advantage, not its tournament structure.

4. Conflation of Optimization Goals: The paper claims to optimize for both low execution time and low variability. However, the "Optimal" or "Oracle" configuration (Figure 10) is determined in an interference-free environment. This is a configuration optimized for peak performance, not robust performance. DarwinGame is solving a different optimization problem—finding the most interference-robust configuration. To claim it is "closer to Oracle" is misleading. The Oracle is not the ground truth for the problem DarwinGame claims to solve. A more valid "Oracle" would be the configuration that, over thousands of trials in a realistic noisy environment, yields the best average performance and lowest variance.

---

Questions to Address In Rebuttal

1. Please provide a principled, non-intuitive justification for the specific three-phase tournament structure (Swiss → Double Elimination → Barrage). Why is this sequence superior to simpler or alternative structures?

2. Justify the formulation of the "Consistency Score." What is the statistical basis for using average(1/rank) as a proxy for performance consistency, as opposed to standard statistical measures like variance or interquartile range?

3. Address the significant threat to internal validity: How can you demonstrate that the configuration selected by DarwinGame is robust to general cloud interference, and not just specifically optimized to perform well under the artificial, self-contention scenario created by co-locating dozens of its own clones?

4. Please clarify the exact protocol used for evaluating the baseline tuners (ActiveHarmony, BLISS, OpenTuner). Specifically, how many times was each parameter configuration sampled to produce a performance measurement in the noisy environment? If this number is lower than the number of "evaluations" a DarwinGame configuration undergoes, please justify why this is a fair comparison.

5. The final selection criteria in the playoffs and final phase (Section 3.5) appear to revert to selecting winners based solely on execution time. How does this square with the stated design goal of finding configurations with low performance variations? How is the trade-off between mean performance and variance explicitly managed in the final decision?

Paper: DarwinGame: Playing Tournaments for Tuning Applications in Noisy Cloud Environments Reviewer Persona: The Synthesizer (Contextual Analyst)

---

Summary

This paper addresses the challenging and highly relevant problem of performance auto-tuning in noisy, interference-prone cloud environments. The authors correctly identify that the fundamental assumption of existing tuners—a stable execution environment for measurement—is violated in shared cloud settings, leading to suboptimal results.

The core contribution is a conceptual shift in how to approach this problem. Instead of seeking an unstable, absolute measure of a configuration's performance, the authors propose finding the most robustly performant configuration through systematic relative comparison. They operationalize this insight with "DarwinGame," a novel framework that runs a multi-stage tournament between different parameter configurations. By co-locating competing configurations in "games," the system ensures they are subject to similar environmental noise, allowing for a fairer assessment of their relative merits. The tournament structure, creatively borrowing established formats like Swiss and double-elimination, serves to progressively identify configurations that are not only fast on average but also resilient to performance variations.

Strengths

1. A Foundational Reframing of the Problem: The most significant strength of this work is its elegant reframing of the tuning problem for shared environments. The insight to abandon the futile search for "true" execution time in a noisy system and instead focus on identifying the "winner" through relative, head-to-head competition is profound. This shifts the paradigm from function optimization to robust selection, which is a much more appropriate model for the target environment. This paper could well be a foundational piece for a new sub-area of research, as the authors suggest in the abstract.

2. High Problem Relevance and Practicality: The problem being solved is not a niche academic curiosity; it is a real, expensive, and increasingly common challenge for developers deploying performance-sensitive applications on the cloud. The paper correctly motivates this need (Section 1), and the proposed solution is designed for direct application in these environments without requiring special, dedicated infrastructure, which is a major practical advantage.

3. Creative and Well-Motivated Methodology: The application of formal tournament structures from competitive game theory is not merely a metaphor; it forms the backbone of the methodology. The choice of different tournament styles for different phases of the search (e.g., Swiss for broad exploration, barrage/knock-out for final selection) is well-justified and demonstrates a thoughtful design process. This interdisciplinary approach is a standout feature of the paper.

4. Connection to Robustness as a First-Class Citizen: The work implicitly connects to the broader field of robust optimization. By incorporating a "consistency score" (Section 3.4, Page 7) and evaluating performance variation as a key metric (Figure 2, Page 3), the authors are not just looking for the fastest configuration, but the most reliable one. In production cloud systems, predictability is often as important as raw speed, and DarwinGame is one of the first tuning systems I have seen to treat this as a primary objective from the ground up.

Weaknesses

While the core idea is excellent, the paper could be strengthened by better situating itself within existing theoretical landscapes and exploring the boundaries of its own design.

1. Missed Opportunity to Formalize the Connection to Robust Optimization: The authors have intuitively developed a system that performs robust optimization, but they do not connect it to the rich literature in that field (e.g., from operations research or machine learning). Framing their "consistency score" and tournament structure in the context of finding a solution that is optimal under uncertainty could add significant theoretical weight. This is less a flaw and more a suggestion to elevate the paper's contribution by connecting it to a more formal foundation.

2. Heuristic-Driven Tournament Design: The design of the tournament contains several "magic numbers" and heuristic choices (e.g., early termination at 25% work completion with a 10% deviation threshold, the number of players per game, the number of regions). While the authors provide empirical justification and some sensitivity analysis, the paper would benefit from a deeper discussion of the principles guiding these choices. It feels like a system that works very well, but the "why" behind some specific design decisions could be further elucidated.

3. Unexplored Scalability to Distributed Systems: The evaluation is confined to multi-threaded applications running on a single (though potentially large) VM. The core mechanism of co-location for fair comparison becomes vastly more complex for distributed applications that span multiple nodes, where both intra-node and inter-node (network) interference exist. This is a key limitation on the scope of the current work that should be more explicitly discussed as an avenue for future research.

Questions to Address In Rebuttal

1. The concept of a "consistency score" is an intuitive proxy for robustness. Could you comment on how DarwinGame relates to the formal field of robust optimization or optimization under uncertainty? Could a more formal definition of robustness potentially guide the tournament structure or scoring, perhaps leading to a more principled design?

2. The tournament structure is quite intricate. How sensitive is the final outcome to the specific choice and parameterization of tournament styles? For instance, what would be the impact of using only a double-elimination format from the start, or of significantly changing the early-termination criteria described in Section 3.3 (Page 6)?

3. Your co-location strategy is key to enabling fair comparisons on a single node. How do you envision the DarwinGame concept extending to distributed applications (e.g., MPI-based HPC workloads or microservices) that run across multiple nodes? What new challenges would arise in trying to create a "fair game" in that scenario?

Review Form: The Innovator

---

Summary

The paper presents DarwinGame, a performance autotuner designed specifically for noisy cloud environments where performance is variable due to interference from co-located tenants. The authors argue that traditional tuners, which assume a dedicated and stable execution environment, perform sub-optimally. The core idea of DarwinGame is to abandon the goal of measuring a configuration's absolute performance and instead focus on its relative performance. This is achieved by co-locating multiple application instances with different configurations and running them concurrently in a "tournament." By subjecting all competitors to the same environmental noise, the system can identify configurations that are not only fast but also resilient to interference. The tournament is a complex, multi-phase structure employing Swiss, double-elimination, and barrage styles to progressively filter and select the winning configuration. The authors claim this is the first performance tuner effective in such shared, interference-prone environments and demonstrate significant reductions in execution time and performance variability compared to existing tuners used "as-is" in the cloud.

---

Strengths

1. Novelty in the Tournament Structure: The primary novel contribution is the design and application of a sophisticated, multi-phase tournament framework for autotuning. While the concept of competitive evaluation is not entirely new (see Weaknesses), the specific synthesis of Swiss, double-elimination, and barrage tournament styles, tailored to different stages of the search process (exploration vs. exploitation), is a novel approach in this domain. The inclusion of a "consistency score" (Section 3.4, Page 7) to explicitly reward low-variability configurations is also a well-integrated and novel feature.

2. Reframing the Objective Function: The paper's conceptual shift from seeking the absolute optimal configuration (as determined in a sterile environment) to finding the most robust and relatively superior configuration within a noisy environment is a key insight. This pragmatic reframing is the intellectual foundation of the work and represents a novel perspective for the autotuning community when addressing cloud deployments.

3. Justification of Complexity via Ablation: The ablation study presented in Figure 16 (Page 12) is a critical strength. It systematically demonstrates that the individual components of the complex tournament design (e.g., the regional phase, the double-elimination format, the use of a consistency score) each contribute to the final performance. This provides strong evidence that the added complexity is not superfluous but is in fact justified by the results, directly addressing a key concern about the trade-off between the complexity of the method and its benefits.

---

Weaknesses

1. Insufficient Differentiation from Prior Art ("Siblingrivalry"): The most significant weakness is the paper's failure to adequately differentiate its core idea from prior art, specifically Ansel et al.'s "Siblingrivalry: online autotuning through local competitions" [12]. The fundamental concept of running different program variants concurrently to compare their relative performance under identical conditions is the central tenet of Siblingrivalry. DarwinGame appears to be a more complex and structured evolution of this same core idea. The authors cite [12] in their introduction but do not provide a direct comparison in the Related Works (Section 6), which is a major omission. Without this, the claim of being the "first" (Abstract) is questionable. The novelty rests entirely on the specific tournament structure, not on the underlying concept of competitive co-location.

2. Overstated Claim of a "New Subarea": The abstract's claim that this work "introduces a new subarea of performance tuning" is an overstatement. Performance tuning in the presence of noise and variability, particularly in shared and cloud systems, is a known and previously studied problem [e.g., 28, 89]. This paper proposes a novel solution to a pre-existing challenge, but it does not establish an entirely new field of study.

3. Borrowed, Not Invented, Mechanisms: The tournament formats (Swiss, double-elimination) are established mechanisms borrowed from game theory and competitive sports. The novelty lies strictly in their application and synthesis for the autotuning problem. The paper should be more precise in its language to reflect that it is presenting a novel application of existing formalisms rather than inventing these mechanisms from first principles.

---

Questions to Address In Rebuttal

1. Please add a detailed paragraph to Section 6 (Related Works) that explicitly compares DarwinGame to Ansel et al.'s Siblingrivalry [12]. What are the key conceptual and algorithmic differences? Is DarwinGame a generalization of Siblingrivalry's "local competitions," and if so, what fundamental limitations of Siblingrivalry does your multi-phase global tournament structure overcome?

2. The tournament design is quite complex. The ablation study shows that the components are beneficial, but it does not fully justify the specific choices. For example, why is a Swiss-style tournament optimal for the initial regional phase, and why must it be followed by a double-elimination phase? Could a simpler, single-format tournament (e.g., a large-scale double-elimination tournament from the start) achieve comparable performance with less design complexity?

3. Please justify or revise the claim of introducing a "new subarea." Could you provide evidence that the problem of tuning in interference-prone environments was not considered a distinct challenge before this work? Otherwise, I suggest rephrasing to state that you are introducing a "novel tournament-based framework for the established challenge of..." to more accurately reflect the contribution.

Title: Debugger Toolchain Validation via Cross-Level Debugging Reviewer: The Guardian

---

Summary

This paper introduces Cross-Level Debugging (CLD), a technique for validating debugger toolchains by comparing execution traces obtained from source-level and instruction-level debugging of the same executable. The core idea is that these two traces, despite their difference in granularity, must adhere to three predefined relations: reachability preservation, order preservation, and value consistency. The authors implement this concept in a tool called DEVIL and evaluate it on GDB and LLDB, reporting the discovery of 27 new bugs, of which 18 have been confirmed or fixed by developers. The work positions itself as an improvement over prior techniques that compare traces from different executables (e.g., optimized vs. unoptimized), which can lead to invalid comparisons.

Strengths

1. The fundamental premise of comparing traces within a single executable is methodologically sound and effectively circumvents a major class of false positives inherent in prior work. The example in Section 6.2 (Figure 5, page 12) provides a convincing demonstration of how comparing different optimization levels can lead to spurious reports, a problem this work correctly avoids.

2. The empirical results are significant. Discovering 18 confirmed issues (including four P2 critical bugs) in mature, widely-used infrastructure like the GCC/GDB and Clang/LLDB toolchains is a substantial practical contribution and provides strong evidence for the technique's efficacy.

Weaknesses

My primary concerns with this work relate to the unexamined assumptions in its core formulation, a superficial comparison with the state-of-the-art, and an underestimation of methodological limitations.

1. Circular Reasoning in Foundational Assumptions: The entire framework rests on the three relations defined in Section 3.1 (page 4). However, the validity of these relations themselves depends on the correctness of the debug information and the debugger's interpretation of it—the very components being tested. Specifically, Relation R#1 (Reachability preservation) assumes that if a source line is reachable, it has a corresponding machine instruction that can be stepped to. This presupposes a reasonably correct mapping in the DWARF information. A compiler bug could easily generate code for a source line but emit faulty or no debug information for it, making it "unreachable" at the source level while being present at the instruction level. The paper is therefore not testing for correctness from first principles, but rather for internal consistency under the assumption that the debug information is not catastrophically broken. This circularity is a fundamental conceptual weakness that is not addressed.

2. Insufficient and Vague Comparison to State-of-the-Art: The comparison against Debug² in RQ4 (Section 5.4, page 10) is unconvincing. Table 5 (page 11) shows that DEVIL finds bugs that Debug² does not, but the authors' explanation is limited to the vague assertion that "DEVIL considers a broader range of program states than Debug²." This is not a scientific analysis. A rigorous comparison would require selecting a specific bug found by DEVIL but not Debug², and providing a mechanistic explanation of why DEVIL's relations (R#1-R#3) trigger a violation while Debug²'s invariants (e.g., hit line consistency, backtrace invariants) do not. Without such a detailed, evidence-based analysis, the claim of complementarity and superiority is unsubstantiated.

3. Downplaying of Manual Effort and Selection Bias: The authors admit in Section 6.1 (page 12) that their process generates false positives, particularly from uninitialized variables, and requires manual inspection to filter. The claim that this is "generally straightforward" and the effort "remains manageable" is anecdotal. The work would be much more rigorous if it quantified this effort. What is the ratio of raw violations to valid bug reports? How much human time per test program is required? This omission masks a potentially significant limitation in the tool's practical automation. Furthermore, the exclusion of programs that take more than 60 seconds to debug (Section 5.5, page 10) introduces a clear selection bias. This methodology explicitly avoids complex, long-running programs where the most subtle and difficult-to-find debugger bugs (e.g., related to memory management, complex state reconstruction) are likely to manifest.

4. Lack of Precision on State Comparison: Relation R#3 (Value consistency) is central to the approach, yet the paper is imprecise about what constitutes the "variable values" being compared. Does this scope include only local variables on the stack? What about global variables, heap-allocated data, and machine registers? Debuggers often perform complex reconstructions for optimized-out variables. The paper provides no details on how DEVIL identifies and compares the full, relevant program state, especially in the face of such DWARF-based value reconstruction. This lack of detail makes it difficult to assess the true technical depth and robustness of the implementation.

Questions to Address In Rebuttal

1. Please defend the foundational R#1 (Reachability) relation against the charge of circularity. How can this relation be considered a reliable oracle when a primary class of compiler bugs involves the generation of incorrect or incomplete DWARF information, which would directly cause the relation to fail?

2. Provide a concrete, step-by-step analysis for at least one of the 13 bugs that DEVIL reportedly found but Debug² could not. You must explicitly detail which of your relations (R#1, R#2, or R#3) was violated and why the program state at that point would not have violated any of the invariants used by Debug².

3. Please quantify the manual effort required to use DEVIL. For the experiments run, what was the total number of violations flagged by the tool, and what percentage of these led to the 27 valid bug reports?

4. Clarify the precise scope of "variable values" checked by R#3. How does your implementation handle variables that are not explicitly in memory at a breakpoint (e.g., enregistered variables, values reconstructed via DWARF expressions)? Does your value comparison account for these complex cases?

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper introduces "Cross-Level Debugging" (CLD), a novel and insightful approach for validating debugger toolchains. The core contribution is a new form of test oracle that avoids the pitfalls of prior work. Instead of comparing the behavior of two different executables (optimized vs. unoptimized), CLD validates a debugger by comparing two different views of a single execution trace: the source-level view (via step) and the instruction-level view (via stepi). The authors posit that for the same executable, these two traces must adhere to a set of consistency properties related to reachability, ordering, and variable values.

The authors implement this idea in a tool called DEVIL and apply it to the GDB and LLDB toolchains. The results are compelling: they identified 27 new bugs, with 18 already confirmed or fixed by developers, including several marked as "P2 critical." This work provides both a new conceptual framework for debugger validation and strong empirical evidence of its practical effectiveness.

Strengths

1. An Elegant and More Robust Oracle: The fundamental contribution of CLD is its reframing of the oracle problem in debugger testing. The prevailing approach, comparing optimized and unoptimized traces, is notoriously brittle. Compiler optimizations can legally and drastically alter code structure, making trace comparison a source of numerous false positives. CLD cleverly sidesteps this by creating a self-referential oracle within a single execution. This is a far more robust foundation, as the relationship between the source and instruction levels of a single compiled binary is more constrained and less subject to the radical transformations that occur between optimization levels. The ability of DEVIL to find numerous bugs at the -O0 level (as shown in Table 2, page 8) is a powerful testament to the weakness of relying on unoptimized traces as a "golden reference."

2. Significant and Immediately Impactful Results: The work's practical significance is beyond doubt. Unearthing 27 bugs (18 confirmed) in mature, critical infrastructure like GCC/GDB and Clang/LLDB is a major achievement. The fact that four of these were deemed "P2 critical" underscores that DEVIL is not just finding cosmetic issues but significant, developer-misleading bugs. This places the work squarely in the tradition of high-impact research that directly improves the tools our entire community relies on.

3. Excellent Positioning and Comparative Analysis: The paper does a good job of placing itself in the context of prior work, particularly its relationship with the state-of-the-art tool Debug² [3]. The comparative evaluation in Table 5 (page 11) is crucial. It shows that the majority of bugs found by DEVIL are not found by Debug², and conversely, that DEVIL can find some bugs discovered by Debug². This is the hallmark of a truly complementary technique. It doesn't just incrementally improve upon an existing method; it provides a new and orthogonal axis for validation, demonstrating that the problem space is richer than previously addressed.

4. A New Conceptual Lens: Beyond its immediate utility, the "Cross-Level" concept is a valuable intellectual contribution. It can be seen as a specific, well-motivated instantiation of differential or metamorphic testing, where the transformation is not on the program's code, but on the level of abstraction used to observe its execution. This idea may be generalizable to other domains where tools provide multiple views of the same underlying artifact (e.g., profilers vs. debuggers, static vs. dynamic analyzers).

Weaknesses

While the work is strong, there are areas where its context and future potential could be explored further. These are not so much flaws as they are opportunities for deeper synthesis.

1. Scope of the Oracle: The three proposed relations—Reachability, Order, and Value consistency—are intuitive and clearly effective. However, they likely do not represent a complete specification of cross-level consistency. For instance, are there potential inconsistencies in the reported call stack structure, type information, or thread states between the two levels? The paper could benefit from a discussion on the potential completeness of their oracle and what other classes of bugs might be missed.

2. Generalizability to Other Programming Paradigms: The paper rightly notes in Section 6.5 (page 13) that applying CLD to interpreted or JIT-compiled languages is a challenge. This is a key boundary for the work's impact. It would be valuable to see a more detailed discussion of what the fundamental obstacles are. For a language like Python or Java, what constitutes the "instruction level"? Is it the bytecode, or the JIT-compiled native code? Each choice presents different conceptual and technical hurdles. Expanding on this would help contextualize CLD within the broader landscape of programming languages.

Questions to Address In Rebuttal

1. Regarding the scope of the oracle: Can the authors comment on other potential cross-level relations they may have considered or observed anecdotally? For example, were there any bugs related to inconsistent call stack depth or function parameter reporting between source and instruction stepping that didn't fit neatly into the three existing relations?

2. Regarding generalizability: Could the authors elaborate on the primary conceptual challenge of applying CLD to a language with a managed runtime like Java? Would comparing source-level stepping with bytecode-level stepping be a viable strategy, and what new classes of bugs might that uncover (e.g., in the JVM's debugger interface)?

3. The manual effort for bug reporting is mentioned in Section 6.1 (page 12). To better gauge the signal-to-noise ratio of DEVIL, could the authors provide a rough estimate of how many unique raw violations were typically produced for a test case that led to one of the 27 confirmed bug reports?

Reviewer: The Innovator (Novelty Specialist)

Summary

This paper proposes a new technique for validating debugger toolchains, termed Cross-Level Debugging (CLD). The central claim to novelty lies in the formulation of the test oracle. Instead of comparing the behavior of a program compiled with optimizations against one without (i.e., comparing two different executables), CLD compares two different execution traces generated from the same executable. Specifically, it uses the fine-grained trace from instruction-level stepping (stepi) as a ground-truth oracle to validate the coarser trace from source-level stepping (step). The authors formalize this relationship with three properties: reachability preservation, order preservation, and value consistency. They implement this concept in a framework called DEVIL and apply it to GDB and LLDB, successfully identifying 27 new bugs, 18 of which have been confirmed or fixed.

Strengths

The primary strength of this paper is the novel and elegant formulation of the test oracle for debugger validation. My analysis of the prior art confirms the authors' assertion that previous academic work in this area, particularly Di Luna et al. [3] and Li et al. [8], has predominantly relied on comparing optimized and unoptimized executables. This prior approach is fundamentally flawed, as compiler optimizations can introduce drastic, non-equivalent changes that make a direct comparison intractable and prone to false positives, a point the authors correctly make in Section 6.2 (page 12).

The proposed CLD concept is a significant advancement because it sidesteps this entire problem. By restricting the comparison to a single executable, the authors eliminate the compiler's optimization strategy as a confounding variable. The core insight—that source-level stepping is merely an abstraction over a more fundamental instruction-level execution—is conceptually simple yet powerful. Using one to validate the other within the same execution context is, to my knowledge, a genuinely new approach for systematic debugger validation in the academic literature.

The value of this novel idea is substantiated by the empirical results. The fact that 11 of the 18 confirmed bugs were found at the -O0 optimization level (Table 2, page 8) is compelling evidence that CLD uncovers a class of bugs that are orthogonal to compiler optimizations and would therefore be missed by prior art that specifically targets optimization-related debug information issues.

Weaknesses

While the core concept is novel, its scope and the novelty of its constituent parts warrant closer scrutiny.

1. Limited Generality of the Core Primitive: The novelty is tightly coupled to debuggers that expose a clear and distinct dichotomy between source-level (step) and instruction-level (stepi) execution. While this is standard for compiled languages like C/C++, the CLD concept may not be directly transferable to other paradigms. The authors briefly acknowledge this limitation for interpreted languages in Section 6.5 (page 13), but the novelty of the paper rests heavily on this specific feature. The contribution is thus more of a point-solution for a specific class of debuggers rather than a universally applicable validation theory.

2. Obviousness of the Formalized Relations: Given the core idea of using stepi to validate step, the three proposed relations (R#1: Reachability, R#2: Order, R#3: Value) are logical, almost self-evident consequences. If a source line is executed, the instructions comprising it must also be executed. The novelty is not in defining these properties, but in being the first to systematically apply them as a debugger oracle. This is a minor point, as the application itself is the contribution, but the formalization part of the work is less of an intellectual leap than the core CLD concept itself.

3. Implicit Assumption of Oracle Correctness: The entire methodology relies on the assumption that the instruction-level stepping (stepi) and state inspection at that level are correct. If stepi itself is buggy (e.g., skips an instruction or misreports a register value), CLD might incorrectly flag the step behavior as faulty. The paper does not discuss this potential failure mode, where the oracle itself is compromised.

Questions to Address In Rebuttal

1. The core idea is elegant and seems almost obvious in retrospect. Could the authors comment on whether this cross-level comparison has been used informally in debugger development and testing, even if not published academically? Is there any prior art in, for example, internal design documents, technical reports, or developer blogs for GDB/LLDB that proposes or uses such a method for internal regression testing?

2. The novelty appears tied to the step/stepi dichotomy. How would the core CLD concept be adapted to environments where this distinction is blurred? For example, in a JIT-compiled language, the "instruction level" might be an intermediate bytecode representation before native code is generated. How would the oracle be defined in such a multi-stage execution environment?

3. The work assumes the instruction-level stepping provides a reliable oracle. Can the authors discuss the case where stepi itself behaves incorrectly? Would CLD be able to detect such a bug, or would it lead to a false positive report against the step command? For example, if stepi incorrectly skips an instruction, CLD might report a violation of R#1 (Reachability) for a source line that step correctly stops at. How does the framework handle a faulty oracle?

Paper Title: Dilu: Enabling GPU Resourcing-on-Demand for Serverless DL Serving via Introspective Elasticity Reviewer: The Guardian

---

Summary

The paper presents Dilu, a serverless DL serving system designed to mitigate GPU fragmentation and improve resource utilization. The authors introduce "introspective elasticity," a concept realized through a 2D co-scaling mechanism that combines fast, intra-instance vertical scaling (adjusting SM quotas) with lazy, inter-instance horizontal scaling (launching/terminating instances). The system is comprised of three main components: a multi-factor profiler to determine resource requirements, a resourcing-complementary scheduler to collocate tasks efficiently, and the adaptive co-scaling mechanism to manage resources dynamically. The authors claim that Dilu significantly reduces GPU fragmentation, increases throughput, and maintains QoS guarantees compared to existing baselines.

Strengths

1. Problem Motivation: The paper correctly identifies a critical and timely problem. GPU fragmentation in serverless DL systems, driven by static allocation policies and workload dynamism, is a well-known issue that erodes the cost-efficiency promises of the serverless paradigm. The motivation presented in Section 1 and Figure 2 is clear.

2. Conceptual Approach: The high-level idea of combining vertical and horizontal scaling (termed "2D co-scaling") is sound. Addressing elasticity at both the intra-instance (fine-grained) and inter-instance (coarse-grained) levels is a logical approach to tackling the multifaceted nature of resource management in this domain.

3. Evaluation Breadth: The authors have conducted an extensive set of experiments in Section 5, comparing Dilu against multiple relevant baselines (Exclusive, MPS, FaST-GS, TGS) across various collocation scenarios (training-inference, inference-inference, etc.) and workload patterns. The inclusion of an ablation study (Section 5.4) is commendable.

Weaknesses

My primary concerns with this paper lie in the insufficient validation of core mechanisms, a lack of clarity on crucial implementation details, and potential overstatement of novelty.

1. Unquantified Overhead of the Core Mechanism: The entire vertical scaling capability of Dilu hinges on the Real-time CUDA Kernel Manager (RCKM), which intercepts CUDA API calls via LD_PRELOAD (Section 3.4.1, page 7). This is a highly invasive technique. The paper, however, provides zero quantification of the intrinsic performance overhead of this interception layer itself. The "Vertical scaling overhead" analysis in Figure 11 is misleadingly titled; it measures the impact of collocation on application performance, not the base overhead of the RCKM framework on a single, unimpeded instance. Without this crucial data point, it is impossible to assess the net benefit of Dilu. A mechanism that saves 10% of resources but introduces a 15% performance penalty is a net loss.

2. Insufficient Differentiation from Prior Art: The RCKM mechanism, as described, appears functionally very similar to the temporal sharing mechanisms in prior work, particularly TGS [47], which is cited and used as a baseline. Both use a centralized manager and client-side interception to control kernel execution. The paper fails to articulate the fundamental technical novelty of its token-based vertical scaling mechanism over these existing approaches. The novelty seems to lie in the control logic (Algorithm 2), but the underlying architecture is not new, and this is not adequately discussed.

3. Lack of Rigor in Profiling Validation: The profiler (Section 3.2, page 5) is the foundation upon which all subsequent scheduling and scaling decisions are built. However, the evaluation of the profiler in Table 2 focuses exclusively on efficiency (i.e., the number of iterations to find a configuration). It presents no evidence of accuracy or optimality. How do the authors validate that the "optimal" configuration found by their Hybrid Growth Search Strategy is indeed the ground-truth optimal, or even close to it? A fast profiler that finds a suboptimal configuration compromises the entire system's performance. This is a critical omission.

4. Vague Description of the Co-Scaling Coordination: The paper claims a key contribution is the adaptive 2D co-scaling, yet the coordination logic between the "fast" vertical scaling and "lazy" horizontal scaling is poorly defined. The description in Section 3.4.2 (page 8) is high-level and relies on arbitrary-seeming window sizes (size=40s) and thresholds (φ_out, φ_in). What happens in a scenario of sustained high load that exceeds the capacity of vertical scaling? How is the "lazy" delay determined, and how does the system avoid severe SLO violations during this delay? The mechanism feels more like a collection of heuristics than a robust, principled control system. The claims of a "smooth transition" are not substantiated by a rigorous explanation of the control loop.

5. Potential for Unfair Baseline Comparisons: While the set of baselines is good, the paper provides insufficient detail on their configuration and tuning. For instance, systems like FaST-GS [19] and TGS [47] have their own internal heuristics and parameters. Were these baselines tuned to perform optimally for the specific workloads used in this evaluation? Without this information, there is a risk that the performance gains attributed to Dilu are partially an artifact of sub-optimally configured baselines. Specifically for TGS, it is a temporal scheduler; its performance is highly dependent on how priorities are assigned to jobs. This is not discussed.

Questions to Address In Rebuttal

The authors must address the following points directly and with concrete data to salvage this submission.

1. Please provide a microbenchmark that quantifies the intrinsic latency and throughput overhead imposed by the RCKM interception library on various CUDA kernel calls, independent of any collocation effects. This should be measured on a single instance running without contention.

2. Beyond the control logic in Algorithm 2, what are the precise technical and architectural novelties of the RCKM mechanism when compared directly to the temporal sharing framework presented in TGS [47]?

3. Please provide evidence for the accuracy of the profiling strategies. For at least one representative model, compare the configuration found by your profiler against a ground truth established by an exhaustive grid search to demonstrate how close to optimal your heuristic gets.

4. Could you provide a more detailed algorithm or state machine diagram that clearly illustrates the coordination logic and state transitions between fast vertical scaling-up and lazy horizontal scaling-out, especially under conditions of sustained high load? Please justify the choice of parameters like the 40s window.

5. The affinity-first scheduling principle (Section 3.3, page 6) aims to reduce the "barrel effect." However, how does this principle avoid creating a different form of fragmentation, where certain GPUs become specialized for specific function types, leaving stranded resources that cannot be used by newly arriving, non-affine functions?

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper introduces "Introspective Elasticity (IE)," a novel two-dimensional co-scaling paradigm designed to address the significant problem of GPU fragmentation in serverless Deep Learning (DL) serving. The authors argue that traditional serverless systems, which rely solely on horizontal scaling (adding/removing instances), are ill-suited for the dynamic and resource-intensive nature of DL workloads, leading to overprovisioning and inefficiency.

The core of their contribution is a system called Dilu, which materializes IE through a cross-layer architecture. Dilu combines: 1. Fast, fine-grained vertical scaling: Dynamically adjusting the GPU compute (SM) quotas allocated to running instances on a sub-second timescale to handle short-term workload bursts. This is managed by a token-based runtime mechanism called RCKM. 2. Lazy, coarse-grained horizontal scaling: Making slower, more deliberate decisions to launch or terminate entire instances to adapt to sustained changes in workload.

This 2D co-scaling approach is supported by a multi-factor profiler to determine optimal resource quotas (<request, limit>) for DL tasks and a resource-complementary scheduler that collocates heterogeneous functions to maximize GPU utilization. The experimental results demonstrate significant improvements over existing approaches, showing reduced fragmentation, higher aggregate throughput, and lower SLO violation rates.

Strengths

1. A Compelling and Timely Core Concept: The paper's central thesis—that serverless DL requires a more sophisticated, two-dimensional elasticity model—is both insightful and highly relevant. The rise of LLMs has made GPU efficiency a first-order concern for cloud providers. The proposed "Introspective Elasticity" provides a clear conceptual framework for solving the impedance mismatch between the slow, disruptive nature of horizontal scaling (with its associated cold starts) and the highly bursty, sub-second reality of inference workloads. The idea of using fast vertical scaling as a first line of defense to absorb bursts is elegant and powerful.

2. Excellent Synthesis of Ideas: This work sits at the crossroads of several research domains—serverless computing, GPU virtualization/sharing, and DL systems scheduling—and does an admirable job of synthesizing ideas from each. It takes the elasticity principle from serverless, combines it with fine-grained temporal GPU sharing mechanisms seen in cluster computing (e.g., TGS, Antman), and applies it to the specific problem of heterogeneous DL task collocation. The result is a cohesive system that is more than the sum of its parts. The paper effectively bridges the gap between high-level serverless orchestration and low-level GPU resource management.

3. Strong Systems Contribution and Implementation: The authors have not just proposed an idea; they have built and evaluated a complete system. The architecture presented in Figure 3 (page 4) is well-reasoned, with clear separation of concerns across the control, scaling, and serving planes. The design of the Real-time CUDA Kernel Manager (RCKM) in Section 3.4.1 (page 7) is a practical approach to implementing dynamic, fine-grained control without modifying the GPU driver. The comprehensive evaluation across various workloads and collocation scenarios provides strong evidence for the system's effectiveness.

Weaknesses

While the work is strong, its positioning and some practical considerations could be strengthened.

1. Positioning of the Core Abstraction: The term "Introspective Elasticity" is new and catchy, but the paper could do more to contextualize it within the broader history of adaptive and autonomic computing. The idea of a multi-dimensional control loop that adjusts resources at different granularities and timescales is not entirely new. The novelty here lies in its specific, cross-layer application to the serverless GPU context. A more nuanced discussion of how IE relates to or evolves from prior concepts in adaptive resource management would help solidify its place in the literature, moving it from a system-specific name to a more generalizable principle.

2. Security Implications in a Multi-Tenant Environment: The paper briefly mentions in Section 3.4.1 (page 8) that it relies on container isolation for security. This is insufficient for a system intended for a multi-tenant serverless platform. The fine-grained temporal sharing of a physical GPU, managed by the RCKM, opens a potential surface for timing-based side-channel attacks between functions owned by different tenants. A malicious function could potentially infer information about a co-located victim's workload by observing perturbations in its own kernel execution latencies. This is a well-known concern in shared resource environments and warrants a more serious discussion of the security model and potential mitigation strategies.

3. Practicality of the Runtime Control Mechanism: The RCKM intercepts CUDA API calls to manage kernel execution via a token system. While clever, this introduces overhead on the critical path of every kernel launch. The paper claims this overhead is "negligible" (Section 5.2, page 11), but this is asserted rather than demonstrated with microbenchmarks. For workloads with very high frequencies of small kernels (a common pattern in some models), this interception and communication overhead could become significant. A more detailed analysis of this overhead would increase confidence in the mechanism's real-world viability.

Questions to Address In Rebuttal

1. Could the authors please elaborate on the conceptual novelty of "Introspective Elasticity" compared to prior work on multi-dimensional or hierarchical resource management in cluster systems? How is IE fundamentally different from, for example, a system that combines cluster-level auto-scaling with node-level CPU frequency scaling or I/O throttling?

2. The RCKM mechanism introduces a control loop into every kernel launch. Can you provide microbenchmark data quantifying this overhead? Specifically, what is the latency added to a single cuLaunchKernel call, and how does this impact the end-to-end performance of workloads characterized by a high frequency of short-duration kernels?

3. Beyond relying on containerization, what is your security model for preventing information leakage between co-located functions from different tenants? Have you considered the potential for timing-based side-channel attacks through the shared RCKM and GPU execution pipeline, and what mitigations might be possible within your framework?

Reviewer: The Innovator (Novelty Specialist)

Summary

The authors present Dilu, a serverless DL system centered around a concept they term "Introspective Elasticity" (IE). The core idea is the coordinated, two-dimensional co-scaling of GPU resources. This is achieved by combining: 1) fast, fine-grained, intra-instance vertical scaling (dynamically adjusting an instance's SM quota) to handle immediate workload fluctuations, and 2) lazy, inter-instance horizontal scaling (adding/removing instances) to manage sustained changes in load. The system is realized through a cross-layer architecture featuring multi-factor profiling, a resource-complementary scheduler, and a real-time CUDA kernel manager (RCKM) for vertical scaling. The paper's central claim is that this holistic co-scaling approach can significantly reduce GPU fragmentation, improve throughput, and guarantee QoS in serverless DL environments, advancing beyond prior art which focuses on only one scaling dimension.

Strengths

The primary strength of this work lies in its novel architectural synthesis. My analysis focuses on the degree to which the core ideas advance the state of the art.

1. Novelty of the Core "Introspective Elasticity" Concept: The central contribution is the co-design of fast, intra-instance vertical scaling (scale-up/down) with lazy, inter-instance horizontal scaling (scale-out/in). This departs from prior serverless systems like INFless [51] or FaST-GS [19], which are limited to horizontal scaling over statically-partitioned GPU resources (via MPS). It also advances beyond prior GPU sharing work, such as TGS [47] or Antman [49], which focuses on single-node resource management (temporal sharing) without integrating it into a broader, cluster-wide serverless auto-scaling framework. The explicit "fast-up, lazy-out" dynamic described in Section 3.4 (pages 7-8) is a direct and novel outcome of this architectural synthesis, designed specifically to mitigate cold-start overheads while maintaining elasticity.

2. Advancement in Dynamic Resource Provisioning: The paper makes a significant leap from the static spatial partitioning of MPS [38], which underpins many existing systems [19, 51], to a truly dynamic vertical scaling mechanism. The RCKM (Section 3.4.1, Figure 6) provides continuous, on-demand adjustment of compute quotas without the overhead or limitations of reconfiguring MPS partitions. While the use of LD_PRELOAD to intercept CUDA calls is not new in itself, its application in a token-based system that dynamically adjusts allocations between competing serverless instances based on kernel execution rates represents a meaningful step forward from discrete time-slicing or static priority-based schemes.

Weaknesses

While the overall architecture is novel, it is important to deconstruct the novelty of its constituent parts to accurately place the work in the context of prior art.

1. Precedent for Component-Level Mechanisms:

   • The mechanism for vertical scaling, a token-based kernel throttling system implemented via LD_PRELOAD (RCKM, Section 3.4.1), is conceptually similar to prior work in temporal GPU sharing. Systems like TGS [47] and other container-based GPU sharing frameworks [20, 52] also employ monitor-and-control architectures to manage kernel execution. The novelty here is not in the interception mechanism itself, but in its specific control logic (driven by Kernel Launch Cycle changes) and its tight integration with the horizontal scaler. This distinction should be made clearer.
   • Similarly, the resourcing-complementary scheduling (Section 3.3) is a well-reasoned application of 2D bin-packing heuristics to the problem of co-locating DL tasks. The principles of affinity and complementarity are known in cluster scheduling. The contribution here is an effective implementation for this specific problem domain, rather than a fundamentally new scheduling theory.

2. Marginal Novelty of Profiling Strategy: The Hybrid Growth Search strategy for profiling (Section 3.2, page 5) is an efficiency improvement over exhaustive search or model-based prediction. While effective, it is an incremental advancement—a clever heuristic for navigating a search space. It supports the main contribution but is not, in itself, a significant conceptual leap. The core novelty of the paper would stand even with a less efficient, brute-force profiling method.

Questions to Address In Rebuttal

1. The vertical scaling mechanism (RCKM) shares conceptual underpinnings with systems like TGS [47] which also timeshare the GPU. Could the authors elaborate on the fundamental differences in the control logic (e.g., token-based vs. priority-based time-slicing) that make Dilu's approach uniquely suited for guaranteeing tight latency SLOs in a serverless DL context?

2. The "lazy" horizontal scaling paradigm is predicated on the vertical scaler's ability to absorb transient workload bursts. What are the empirical limits of this absorption capability? At what point does a workload burst become too large or sustained for vertical scaling to handle alone, forcing a reactive (non-lazy) horizontal scale-out and potentially negating some of the claimed benefits of avoiding cold starts?

3. The paper argues that its dynamic vertical scaling is superior to the static partitions of MPS. However, MPS provides strong performance isolation between processes. What, if any, performance interference (e.g., memory bandwidth contention, L2 cache pollution) was observed between co-located instances under Dilu's dynamic token-issuing scheme, and how does the system mitigate it beyond throttling SM access? Is the isolation provided by containers sufficient?

Paper Title: D-VSync: Decoupled Rendering and Displaying for Smartphone Graphics Reviewer The Guardian (Adversarial Skeptic)

---

Summary

The paper proposes D-VSync, a rendering architecture that decouples frame rendering from the display's VSync signal. The core idea is to pre-render frames during idle periods created by computationally "short" frames and store them in an enlarged buffer queue. This buffer of pre-rendered frames is then consumed by the display, theoretically masking the latency of computationally "long" frames and thus preventing stutters. The system consists of a Frame Pre-Executor (FPE) to manage the pre-rendering schedule and a Display Time Virtualizer (DTV) to provide a future timestamp for rendering animations correctly. The authors implement and evaluate D-VSync on OpenHarmony and Android, claiming significant reductions in frame drops and user-perceptible stutters with minimal overhead.

---

Strengths

1. The work addresses a persistent and important problem in mobile graphics: UI stutter caused by workload fluctuations that overwhelm the fixed time budget of a VSync interval.
2. The implementation of the proposed system on two major mobile operating systems (OpenHarmony and AOSP) demonstrates a significant engineering effort and lends a degree of real-world validity to the concept.

---

Weaknesses

My analysis of this paper reveals several significant weaknesses in its foundational claims, methodology, and evaluation that undermine the credibility of its conclusions.

1. Unsupported Foundational Claims: The paper's motivation rests on claims that are presented without sufficient evidence.

   • The assertion that frame rendering time follows a "power law distribution" (Section 1, page 2; Section 3.4, page 5) is a strong statistical claim. However, the only evidence provided is the qualitative shape of the CDF in Figure 1. There is no statistical test (e.g., a goodness-of-fit test) or parameter estimation to justify this specific distribution over other heavy-tailed distributions. This appears to be an assertion based on observation rather than rigorous analysis.
   • The entire applicability of the technique hinges on the claim that 85% of frames are from "deterministic animations" and thus pre-renderable (Figure 9, page 6). The methodology for arriving at this critical 85% figure is completely absent. Without understanding how this data was collected, across which users, apps, and usage patterns, this number is unsubstantiated and may not be generalizable.

2. Critically Flawed Evaluation of Games: The evaluation of mobile games is presented as a simulation, not a real-world implementation (Section 6.1, page 10). The authors state they "use scripts to simulate the D-VSync decoupled pre-rendering pattern" based on collected runtime traces. This is a major methodological flaw. A simulation cannot capture the complex interplay of a real system, including OS scheduling, memory contention, cache effects, and thermal throttling that would be affected by shifting the rendering workload. The impressive results shown in Figure 14 are therefore theoretical at best and cannot be considered proof of the system's effectiveness in the most demanding graphics scenarios.

3. Ambiguous and Potentially Misleading Latency Analysis: The paper claims a significant reduction in rendering latency by 31.1% (Section 6.3, page 11). This is deeply counter-intuitive. A system that intentionally buffers more frames should, by definition, increase the average latency from input to display. The paper clarifies that it reduces the lengthened latency that occurs due to "buffer stuffing" after a frame drop in the baseline VSync architecture. This is a narrow and self-serving definition of latency improvement. The paper fails to present the more critical trade-off: in a scenario with no dropped frames, what is the latency penalty of D-VSync compared to a standard triple-buffered VSync system? By displaying pre-rendered (and therefore older) frames, D-VSync must inherently increase latency in the steady state. This crucial aspect is not measured or discussed, making the latency claims misleading.

4. Unconvincing Solution for Interactive Content: The proposed solution for interactive frames, the Input Prediction Layer (IPL), is described in vague terms such as "curve fitting" and "simple heuristic curves" (Section 4.6, page 8). The case study in Section 6.5 uses a simplistic "linear line fitting" for a complex zoom gesture. The paper provides no evaluation of the prediction accuracy of the IPL, the visual artifacts or user experience degradation when predictions are wrong, or the computational overhead of the prediction itself. Without this, the IPL is an underdeveloped idea, not a validated solution for the 10% of interactive frames the authors identify.

5. Weak Subjective Evaluation: The reduction in "user-perceived stutters" (Section 6.2, page 10) is based on reports from an internal "professional user experience (UX) team." While such evaluations can be useful, the methodology lacks the rigor expected in an academic paper. Key details are missing: Was the study conducted in a blinded manner? How many evaluators were involved? What was the inter-rater reliability? Without these controls, the results are anecdotal and susceptible to confirmation bias.

---

Questions to Address In Rebuttal

The authors must provide clear and concise answers to the following questions to justify the claims made in this paper.

1. Can you provide rigorous statistical evidence to support the claim that frame rendering times follow a "power law distribution," beyond the qualitative CDF plot in Figure 1?
2. Please provide a detailed methodology for how the workload characterization in Figure 9 was performed. How was the 85% figure for "deterministic animations" derived, and how can we be sure it is representative of general smartphone use?
3. Why was the evaluation for games (Figure 14) performed as a simulation instead of a real-world implementation via the proposed custom APIs? How can you defend the validity of these simulated results given that they ignore real-world system dynamics?
4. Please clarify the latency measurement. What is the average end-to-end latency (e.g., from input event to photon) of D-VSync compared to the VSync baseline in a steady-state scenario where no frames are dropped? Is it not true that D-VSync's buffering mechanism necessarily increases this latency?
5. What is the measured prediction accuracy of the Input Prediction Layer (IPL)? What is the user-perceptible impact when an input is mispredicted, and how frequently does this occur in your tested scenarios?

Paper Title: D-VSync: Decoupled Rendering and Displaying for Smartphone Graphics Reviewer Persona: The Synthesizer (Contextual Analyst)

---

Summary

This paper introduces D-VSync, a novel rendering architecture for smartphone operating systems designed to mitigate frame drops and reduce rendering latency. The core contribution is the decoupling of the rendering execution pipeline from the fixed-cadence display refresh cycle (VSync). The authors correctly identify the central conflict in modern graphics stacks: the fluctuating, bursty nature of rendering workloads clashes with the rigid, periodic deadlines imposed by VSync.

The key insight is to leverage the computational time saved during the rendering of simple, "short" frames to create a time buffer for the inevitable, complex, "long" frames. This is achieved through two primary mechanisms: a Frame Pre-Executor (FPE) that proactively renders frames ahead of their display time, and a Display Time Virtualizer (DTV) that provides these future frames with the correct timestamp to ensure animations proceed smoothly and correctly. This architectural change allows the system to build a queue of pre-rendered frames, which can be consumed by the display to hide the latency of a long frame that misses its original deadline.

The evaluation is extensive, covering multiple commercial devices and operating systems (OpenHarmony and AOSP). The results are highly impressive, demonstrating a ~73% reduction in frame drops and a ~31% reduction in latency with negligible power overhead. The fact that this system has been integrated into a commercial product (HarmonyOS NEXT) provides a powerful validation of its practicality and impact.

Strengths

1. Addresses a Fundamental and Increasingly Urgent Problem: The paper tackles a foundational aspect of modern user interfaces. The VSync-based architecture, which has been the cornerstone since "Project Butter" in 2012, is showing its age. The authors provide a compelling analysis in Section 3 (pages 4-5) of how rising screen resolutions, refresh rates, and visual complexity have pushed this architecture to its breaking point. This work isn't solving a niche issue; it is proposing a successor to a decade-old industry standard.

2. Elegant and Well-Motivated Core Concept: The central idea of D-VSync is an elegant piece of systems thinking. It reframes the problem from "how can we make every single frame render faster?" to "how can we design a system that is resilient to frames that are slow?" The motivation, grounded in the observed power-law distribution of frame rendering times (Figure 1, page 2), is clear and convincing. This is a classic application of buffering to smooth out a producer-consumer relationship where the producer (the renderer) has variable performance.

3. Demonstrates Strong System-Level Thinking: The authors show a mature understanding of the problem space. They don't just present an algorithm in isolation; they present a system architecture. The consideration of how D-VSync interacts with orthogonal technologies like LTPO variable refresh rate screens (Section 5.3, page 9) and the provision of dual-channel APIs for both oblivious and aware applications (Section 4.5, page 8) are hallmarks of a well-designed system intended for real-world deployment.

4. Exceptional and Highly Convincing Evaluation: The evaluation is a major strength. The use of both objective (frame drop counters on 75 OS use cases) and subjective (UX expert evaluations) metrics provides a holistic view of the improvements. The breadth of testing across different devices, OSes, and graphics backends demonstrates robustness. The deployment in HarmonyOS NEXT is the ultimate validation, moving this work from an academic curiosity to a proven industrial innovation.

Weaknesses

While this is an excellent paper, its positioning could be strengthened by drawing broader connections to established computer science principles. These are not flaws in the work itself, but opportunities to better frame its significance.

1. Understated Connection to Classic CS Concepts: The authors correctly identify related work in mobile systems, but the core idea of D-VSync has strong parallels in other domains that could be highlighted to generalize the contribution. The system is essentially an implementation of a bounded buffer for a real-time producer-consumer problem. The challenges it solves are conceptually similar to managing jitter in network streaming (using a jitter buffer) or smoothing I/O operations in an operating system (using a disk cache). Explicitly drawing these parallels would elevate the work's contribution from a "graphics system trick" to an application of a fundamental computer science pattern to the domain of mobile graphics.

2. Limited Exploration of Failure Modes and System Dynamics: The paper convincingly shows when D-VSync works, but could benefit from a deeper analysis of when it doesn't. The QQMusic example (Analysis, page 10) is a good start, showing that a stream of very long frames can deplete the buffer and defeat the system. It would be valuable to characterize this boundary condition more formally. For example, how does the system behave under sustained heavy load versus sporadic heavy load? What is the recovery process after the buffer is fully drained? A discussion of these dynamics would add depth.

3. The "Deterministic" Assumption: The approach's effectiveness for legacy apps hinges on the ability to identify "deterministic" animations (claimed as 85% of frames in Section 4.2, page 7). While this is plausible for standard OS transitions, this assumption could be fragile. A deeper discussion on what makes a frame non-deterministic (e.g., dependencies on asynchronous network events, complex user input) and how the system gracefully falls back to standard VSync would be beneficial.

Questions to Address In Rebuttal

1. Could you elaborate on the mechanism for handling unexpected, high-priority events that invalidate the pre-rendered frames? For example, if several frames for a scrolling animation are pre-rendered, but the user suddenly taps a button, these frames must be discarded. What is the performance cost of flushing this buffer, and how does the system quickly pivot to rendering the new state?

2. The case of QQMusic, where performance gains were limited, is very insightful. Can you further characterize the workloads where D-VSync's benefits are diminished? Is it purely a function of the number and duration of consecutive long frames, or are there other factors, such as memory bandwidth contention from having more buffers?

3. The extensible Input Prediction Layer (IPL) is a promising concept. Do you envision this evolving to use more sophisticated machine learning models for prediction, similar to the work cited in VR motion prediction [40]? Could this IPL framework be generalized to predict not just user input, but also other asynchronous events (e.g., network data arrival) to further expand the scope of pre-renderable frames?

Review Form: The Innovator

Summary

The paper proposes D-VSync, a novel rendering architecture for smartphone operating systems designed to mitigate frame drops and reduce latency. The core idea is to decouple the rendering execution from the periodic VSync display event. This is achieved by proactively pre-rendering frames into an enlarged buffer queue, a process the authors term the "accumulation stage." The key claimed novelty lies in the synthesis of two main components: a Frame Pre-Executor (FPE) that schedules frame rendering ahead of time, and a Display Time Virtualizer (DTV) that provides a future timestamp to the application and render service. This allows pre-rendered frames to contain content that is correct for their future display time, not their current execution time. The system thereby allows the computational time saved by short, simple frames to be "banked" and used to absorb the cost of subsequent long, complex frames, smoothing out workload fluctuations.

Strengths

The primary strength of this work from a novelty perspective is its specific, well-engineered synthesis of existing concepts into a new architecture tailored for a persistent problem in smartphone graphics.

1. Novel Architectural Synthesis: The central architectural pattern of combining an aggressive pre-rendering scheduler (the FPE) with a mechanism to ensure temporal correctness for pre-rendered frames (the DTV) is a novel contribution in the context of general-purpose mobile OS rendering pipelines. While buffering is not new, this system moves beyond passive buffering (like triple-buffering) to an active, predictive pre-rendering model.
2. The Display Time Virtualizer (DTV): The DTV is the most significant novel component. The idea of rendering a frame not for "now" but for a calculated point in the future is a clever solution to the correctness problem that would otherwise plague any pre-rendering scheme for dynamic content like animations. It effectively creates a "future-in-software" for the rendering logic to target.
3. Problem-Specific Application: While predictive execution exists in other domains, the authors have identified a specific niche—deterministic UI animations in VSync-bound systems—where such a technique can be highly effective. The insight that the vast majority of frames in UI interactions are deterministic (as claimed in Section 4.2, page 6) provides a strong foundation for the novelty of this targeted approach.

Weaknesses

My main concerns regarding novelty center on the paper's positioning relative to prior art in conceptually similar domains. The individual building blocks of D-VSync are not, in isolation, entirely new.

1. Conceptual Overlap with Speculative/Predictive Execution: The core principle of executing work ahead of time based on a prediction of a future state is well-established. Cloud gaming systems like Outatime [46] use input prediction and speculative frame rendering to hide network latency. Web computing systems like PES [36] proactively schedule work based on anticipated user interactions. While the authors cite these works, the paper could do more to sharply delineate its contribution. The "delta" seems to be in the trigger (deterministic animation vs. user input/network) and the target (VSync jitter vs. network latency), but the fundamental "render-ahead" pattern is conceptually homologous.
2. Limited Novelty of the Input Prediction Layer (IPL): The IPL, described as an extension for interactive scenarios in Section 4.6 (page 8), appears to be a direct application of standard input prediction and curve-fitting techniques. Its novelty lies in its integration into the D-VSync framework, not in the prediction mechanism itself. This is functionally identical to the prediction models cited in the related work for VR [40] and cloud gaming [46].
3. Amortization as a Known Technique: The high-level insight that "sporadic long frames [can] utilize the computational power saved by common short frames" (Abstract, page 1) is a classic description of amortization. The novelty here is not the concept of amortization, but the specific mechanism (FPE+DTV) built to enable it within the rigid constraints of a VSync-driven pipeline. The paper should be careful to frame its novelty in the mechanism, not the high-level concept.

Questions to Address In Rebuttal

1. Please explicitly detail the delta between the Display Time Virtualizer (DTV) and the predictive mechanisms in cloud gaming systems (e.g., Outatime [46]). Is the DTV simply a time-offset calculator based on buffer depth and VSync period, or does it incorporate more complex models of the rendering pipeline state? The novelty of D-VSync hinges on this component being more than a trivial calculation.
2. The approach's effectiveness is predicated on the claim that a large fraction (85%) of frames derive from deterministic animations (Section 4.2, page 6). How does this core assumption, which enables the pre-rendering approach, hold up in emerging UI paradigms that are less deterministic? For example, UIs with heavy physics-based interactions, live data feeds, or on-screen generative AI content. Is the novel contribution fundamentally tied to the animation patterns of today's UIs?
3. There appears to be an inherent tension between the goal of the core D-VSync architecture (building a buffer of frames, which can increase the glass-to-photon latency for any new event) and the goal of the IPL extension (reducing latency for touch interactions). How does the system arbitrate this trade-off? For instance, if the buffer is full of pre-rendered animation frames and the user suddenly provides a new touch input, are the pre-rendered frames discarded? A clearer explanation of this interaction would strengthen the claim of a cohesive novel architecture.

Reviewer: The Guardian (Adversarial Skeptic)

---

Summary

The authors present OMEN, a framework for early termination in Hyperdimensional Computing (HDC) classifiers. The central idea is to reframe the HDC inference process, specifically the distance calculation between a query and class hypervectors, as a statistical sampling problem. By treating the element-wise distance contributions as samples from an underlying distribution, the authors apply sequential statistical tests—specifically, Wald's test with a Holm-Bonferri correction for multiple comparisons—to determine if a classification decision is statistically significant at intermediate points of the computation. The stated goal is to achieve significant inference speedups by processing only a fraction of the hypervector dimensions, while providing a user-configurable statistical bound (α) on the resulting accuracy loss. The method is evaluated across 19 benchmarks, comparing against an unoptimized baseline and several heuristic-based early termination methods.

Strengths

1. Principled Approach to Early Termination: The paper’s primary strength is its attempt to replace ad-hoc heuristics for early termination with a more formal, statistically-grounded methodology. This is a commendable direction for the field.
2. Comprehensive Empirical Validation: The experimental setup is thorough. The evaluation covers four different datasets, three distinct HDC training algorithms (OnlineHD, LeHDC, LDC), and both major HDC variants (BSC, MAP). This breadth lends credibility to the empirical results and demonstrates the method's general applicability across different HDC configurations.
3. Robustness Analysis: The inclusion of experiments with a hardware noise model (Section 7.3, page 14) is a valuable contribution. It demonstrates that the statistical assumptions of OMEN are resilient to the kind of noise one might expect on emerging, error-prone hardware, strengthening the paper's practical claims for edge computing.

Weaknesses

My primary concerns with this submission relate to the rigor of its core theoretical claims and the downplaying of critical limitations.

1. Fundamental Disconnect Between Theory and Application: The entire framework rests on the validity of Wald's test for this problem. Wald's test, like many similar parametric tests, relies on the Central Limit Theorem (CLT), which requires the samples to be independent and identically distributed (iid) or satisfy weaker conditions that still lead to normality. The authors initially build their "Statistical View of HDC" (Section 4, page 5) on an iid assumption. However, they correctly concede in Section 6 (page 8) that advanced, state-of-the-art training algorithms (like LeHDC and LDC) violate this iid assumption by introducing correlations between hypervector elements.

   The paper pivots to the weaker condition of "exchangeability." However, the justification for why Wald's test remains valid for exchangeable variables is insufficient. Section 6.3.2 (page 11) states that CLT holds for exchangeable variables if the covariance is zero (which is not shown to be the case here) or that they can be "approximated with conditionally iid random variables." This approximation comes with a bound that is "loose" and is not analyzed. The authors ultimately fall back on an empirical claim: "empirically, we find the Wald's test effectively and robustly bounds the accuracy loss." This is a critical weakness. A framework claiming to provide "statistical guarantees" cannot be based on a statistical test whose core assumptions are not met and whose applicability is justified only by empirical observation. The "guarantee" is therefore not a formal one.

2. Overstated "Accuracy Guarantee": The paper repeatedly uses the term "accuracy guarantees," stating that the accuracy drop is bounded by α (e.g., acc(N) - accomen(N,a) ≤ a in Section 5.3.2, page 8). This is a misrepresentation. The parameter α in a hypothesis testing framework is a bound on the probability of making a Type I error (i.e., falsely rejecting a true null hypothesis). It is not a direct, deterministic bound on the degradation of the classifier's accuracy metric over a test set. While the two are related, the paper fails to formally derive the relationship and instead treats them as equivalent. This conflation of statistical significance with classification accuracy is a recurring issue and weakens the paper's central promise of providing rigorous, well-defined guarantees.

3. Diminishing Returns on Optimized Models: The evaluation reveals a crucial limitation in Section 7.2.1 (page 13) regarding the LDC-BSC benchmarks. For these models, which use highly compressed hypervectors (N=256), the speed-up from dimensionality reduction is almost entirely negated by the computational overhead of the statistical tests. The fitted line SUR = 0.239 * DRR + 0.669 shows that a 4x dimension reduction (DRR=4) would yield a speedup of only ~1.6x, far less than the near-linear speedups seen on other models. This is not a minor point; LDC is a state-of-the-art algorithm designed for creating compact, efficient models. The fact that OMEN's benefits severely diminish on precisely these kinds of highly optimized models calls into question its utility for the most resource-constrained scenarios, which are a primary target for HDC. This limitation is presented as a mere implementation detail (floating-point overhead) rather than a fundamental scaling issue of the proposed method.

Questions to Address In Rebuttal

The authors should address the following points to strengthen their submission:

1. Please provide a more rigorous theoretical justification for the application of Wald's test to the exchangeable element-wise distance vectors produced by learned HDC models. If relying on an approximation via De Finetti's theorem, please provide an analysis of the approximation error and how it impacts the validity of the claimed α-bound. Without this, the "statistical guarantee" is unsubstantiated.
2. Please clarify the precise nature of the "accuracy guarantee." Is it a hard bound on the observed accuracy drop on a finite test set, or is it a probabilistic bound on the Type I error of the underlying statistical test? If the latter, please provide a formal argument connecting this statistical error rate to the expected drop in classification accuracy. Please consider rephrasing "guarantee" to more accurately reflect its statistical meaning.
3. The limited speedup on LDC-BSC models is a significant concern for the practical applicability of OMEN. Could the authors provide a more detailed analysis of the overhead? Is the floating-point-to-integer conversion proposed in Section 7.2.1 a tested solution or speculation? Please comment on the scalability and relevance of OMEN for future HDC models that are likely to be even more compressed and optimized.
4. The selection of termination points (e.g., powers of two or regular intervals) appears to be a manually-tuned hyperparameter. How sensitive are the results (both speedup and accuracy) to the choice and frequency of these points? Is there a principled method for selecting them, or does this introduce another layer of heuristic tuning that the paper aims to avoid?

Paper Title: Early Termination for Hyperdimensional Computing Using Inferential Statistics Reviewer Persona: The Synthesizer (Contextual Analyst)

---

Summary

This paper presents OMEN, a novel framework for performing principled early termination during inference in Hyperdimensional Computing (HDC) classifiers. The central and most significant contribution is the reframing of HDC's computational process—specifically, the element-wise distance calculation—as a problem of sequential statistical sampling and hypothesis testing. By treating the elements of a hypervector not as fixed data points but as samples drawn from an underlying distribution, the authors are able to apply classical inferential statistics (namely, Wald's test with a Holm-Bonferri correction) to determine, at intermediate points, whether a classification decision has been reached with a desired level of statistical confidence.

This approach elegantly replaces the ad-hoc, heuristic-based methods common in prior work with a statistically grounded mechanism that provides a formal upper bound on the accuracy loss (α). A key theoretical insight is the connection the authors draw between the "fully distributed" property of HDC and the statistical property of "exchangeability," which they compellingly argue (and prove for several modern training algorithms in Section 6) is the necessary condition for their framework to apply. The work is supported by a thorough empirical evaluation across 19 benchmark configurations, demonstrating significant speedups over baseline and heuristic methods while robustly maintaining the promised accuracy guarantees, even in the presence of simulated hardware noise.

Strengths

1. A Powerful Conceptual Bridge: The paramount strength of this paper is the conceptual leap it makes. It connects two seemingly disparate fields: the brain-inspired, algebraic world of Hyperdimensional Computing and the rigorous, probabilistic world of inferential statistics. The formulation of HDC inference as a statistical sampling problem (Section 4) is a beautiful and potent idea. It provides a formal language to describe what HDC practitioners have long understood intuitively—that information is distributed redundantly across the vector and that partial computations yield meaningful approximations. This work gives that intuition a solid mathematical and statistical foundation.

2. From Heuristics to Principled Guarantees: The most significant practical impact of this work is that it moves the optimization of HDC inference from the realm of heuristics to that of principled, tunable guarantees. The ability for a system designer to specify a confidence level (α) and be assured that the accuracy degradation will not exceed this bound is a massive step forward for deploying HDC in mission-critical or reliability-sensitive edge applications. This is a sign of a maturing field, moving from clever tricks to robust engineering.

3. Theoretical Justification for Modern HDC: The paper could have stopped at applying its method to simple, i.i.d. HDC models. Instead, the authors' treatment of exchangeability in Section 6 is a crucial and highly commendable piece of work. By demonstrating that advanced training algorithms (like distance-based iterative training and gradient descent-based LDC) preserve exchangeability, they vastly broaden the applicability and relevance of their OMEN framework. It shows that this statistical view is not just a curiosity for "vanilla HDC" but a fundamental property that persists even as models become more complex and optimized.

4. Thorough and Convincing Evaluation: The experimental validation in Section 7 is comprehensive. The authors test their approach across multiple datasets, with both binary (BSC) and real-valued (MAP) HDC variants, and against a spectrum of training algorithms from simple (OnlineHD) to state-of-the-art (LeHDC, LDC). The inclusion of multiple heuristic baselines and the clear presentation on Pareto-frontier plots (Figure 6) effectively showcases the superiority of their method. The robustness check against bit-flip errors (Section 7.3) is particularly compelling, as it demonstrates the framework's natural resilience and suitability for emerging noisy hardware platforms.

Weaknesses

As a synthesizer, I see "weaknesses" more as unexplored frontiers or necessary boundary conditions of the proposed idea.

1. Overhead in Highly Compressed Regimes: The evaluation honestly reveals a key limitation: the computational overhead of the statistical tests can become non-trivial for HDC models that are already highly compressed (e.g., LDC-BSC with N=256, as discussed in Section 7.2.1). In these cases, the performance gains from early termination are marginal or even negative. This suggests that OMEN is most impactful when there is "room to optimize"—i.e., on larger hypervectors. This isn't a flaw so much as a characterization of the algorithm's application domain, which could be stated more explicitly.

2. Scope of HDC Variants: The work focuses on dense HDC models (BSC and MAP). The broader HDC landscape includes other important variants, such as sparse binary representations or Fourier Holographic Reduced Representations (HRRs) using complex numbers. It is not immediately obvious how this statistical framework would translate to these other domains, especially sparse models where the i.i.d. or exchangeability assumptions might be structured differently.

3. Selection of Termination Points: The OMEN algorithm takes a pre-defined list of term_points as input. The paper does not delve into the strategy for selecting these points. Are they chosen linearly? Logarithmically? Is there an optimal, data-driven way to select these checkpoints to maximize the probability of early termination while minimizing the overhead of frequent statistical tests? This represents an important, unaddressed hyperparameter in the OMEN framework.

Questions to Address In Rebuttal

1. Could the authors elaborate on the applicability of their statistical view to other HDC paradigms, particularly sparse HDC? Would the underlying assumption of exchangeability hold, or would a different statistical formulation be required?

2. Regarding the overhead in low-dimensional cases (e.g., LDC-BSC), have the authors considered approximations for the statistical tests themselves (e.g., using fixed-point arithmetic or lookup tables for the CDF inverse) to make OMEN more viable for extremely resource-constrained scenarios where even a few floating-point operations are expensive?

3. What is the sensitivity of OMEN's performance to the choice and frequency of termination points? Could the authors provide some intuition or preliminary results on how one might develop a strategy for selecting these points for a given application?

4. This is a more speculative, forward-looking question: Now that you have established this powerful statistical lens for viewing HDC inference, have you considered if it could be applied to other areas of the HDC pipeline? For instance, could this statistical significance framework be integrated into the training process itself to guide model learning or determine an optimal vector dimensionality for a given task?

Reviewer: The Innovator (Novelty Specialist)

---

Summary

This paper presents OMEN, a framework for dynamic early termination of inference in Hyperdimensional Computing (HDC) classifiers. The central claim of novelty rests on a new conceptual model: reframing the HDC inference process, specifically the element-wise distance calculation, as a statistical sampling and sequential hypothesis testing problem. By treating each dimension's contribution to the final distance metric as a sample from an underlying distribution, the authors apply established statistical methods—namely Wald's sequential probability ratio test with a Holm-Bonferri correction—to terminate inference once a classification decision reaches a user-specified level of statistical significance (α). A secondary novel contribution is the formal justification that this statistical view holds not only for simple i.i.d. hypervectors but also for vectors produced by advanced training algorithms, by proving that these algorithms preserve the property of statistical exchangeability.

---

Strengths

The primary strength of this paper lies in its genuinely novel conceptual contribution. My analysis confirms the following points of novelty:

1. The "Statistical View" of HDC Inference: The core novelty is the reframing of HDC distance computation as a statistical testing problem, as detailed in Section 4 (page 5). While the probabilistic nature of HDC is well-understood, to my knowledge, no prior work has explicitly modeled the dynamic, dimension-by-dimension accumulation of distance as a sequential sampling process to be terminated via a formal statistical test. This is a significant conceptual leap.

2. Principled vs. Heuristic Termination: Prior art on early termination in HDC, such as Chuang et al. [9] and Imani et al. [30], relies on heuristic, threshold-based methods (e.g., "is the margin between the top two classes greater than a fixed value?"). This work is the first to replace these heuristics with a principled statistical framework. The ability to provide an explicit upper bound on the accuracy loss (α) is a direct result of this novel approach and represents a clear advancement over the state-of-the-art.

3. Formalization via Exchangeability: The paper's claim of novelty is substantially reinforced by Section 6 (page 8). The intuition that information in HDC is "fully distributed" is not new. However, this work is the first to formally connect this property to the statistical concept of exchangeability and, more importantly, to prove that modern HDC training algorithms (e.g., LeHDC [14], LDC [15]) produce hypervectors that satisfy this property. This theoretical result is a novel contribution that extends the applicability of the core "statistical view" beyond the trivial i.i.d. case, giving the work a solid theoretical foundation.

---

Weaknesses

From the perspective of novelty, the weaknesses are not in the core idea but in ensuring the boundaries of the contribution are precisely defined.

1. Novelty is in Application, Not Invention: The statistical machinery employed—Wald's test [70] and the Holm-Bonferri method [28]—are foundational statistical tools developed decades ago. The paper is clear about this, but it must be emphasized that the novelty is entirely in the identification of a new problem domain (HDC inference) that can be mapped to these methods, and the development of the conceptual framework ("statistical view") that makes this mapping sound.

2. Delta Over Prior Art Could Be Sharper: The conceptual delta between "fully distributed" and "exchangeable" could be articulated more sharply. While the formal proof is new, the practical implication—that dimensions are symmetric and permutable without loss of information—is a long-standing intuition in the HDC community. The paper would be stronger if it explicitly stated that the novelty is the formal proof of preservation of this property under learning, rather than the discovery of the property itself.

---

Questions to Address In Rebuttal

1. The core statistical methods you employ are well-established in many fields, from manufacturing quality control to clinical trials and online A/B testing. Can you confirm that your claim to novelty is exclusively centered on the "statistical view of HDC" (Section 4) that enables the application of these tests, and the formal exchangeability proof (Section 6) that justifies it for learned models?

2. Regarding Section 6, the concept of permutation invariance is fundamental to the robustness of HDC. Could you elaborate on the novel insight gained from framing this as "exchangeability," beyond providing the necessary theoretical rigor for applying Wald's test? Does this new framing open up other avenues for analysis or optimization in HDC that were not apparent before?

3. Have analogous sequential testing approaches been applied in conceptually adjacent domains, such as terminating distance calculations early in high-dimensional nearest-neighbor search using paradigms like Locality-Sensitive Hashing (LSH)? A brief discussion of what makes this application to HDC uniquely novel (perhaps due to the specific algebraic structure or the element-wise nature of its operators) would further strengthen the paper's contribution.

Review Form: Earth+: On-Board Satellite Imagery Compression Leveraging Historical Earth Observations Reviewer: The Guardian

Summary

The authors present Earth+, a system for on-board satellite imagery compression that leverages historical images from an entire satellite constellation to serve as reference frames. The core idea is to identify and downlink only the "changed" tiles in a newly captured image relative to a fresh, cloud-free reference image sourced from any satellite in the constellation. This reference is uplinked to the target satellite after significant compression. The authors claim this approach reduces downlink usage by up to 3.3x compared to state-of-the-art methods without sacrificing image quality, and they evaluate this on the Sentinel-2 and Planet datasets.

However, the work's core premise is undermined by a critical methodological flaw: its evaluation on pre-processed, ground-aligned public datasets. This experimental design choice sidesteps the most significant real-world challenges of multi-satellite change detection—namely geometric misalignment, sensor noise, and complex radiometric differences—rendering the reported performance gains suspect and likely unachievable in an operational environment.

Strengths

1. The paper addresses a well-known and critical problem in Earth observation: the severe bottleneck in satellite-to-ground downlink capacity.
2. The core concept of leveraging the higher temporal revisit rate of a constellation to obtain fresher reference images is, in principle, a sound and logical direction for improving reference-based compression.
3. The paper is clearly written and well-structured, making the proposed system and its evaluation easy to follow.

Weaknesses

My primary concerns with this submission center on the validity of the experimental setup and the robustness of the proposed techniques. The work, in its current form, appears to solve a simplified version of the problem that does not exist in practice.

1. Critical Flaw: Evaluation on Processed, Non-Raw Imagery. The single most significant weakness of this paper is the use of public Sentinel-2 (likely Level-1C or 2A) and Planet (Level 3B) datasets for evaluation. These products have already undergone significant ground-based post-processing, including precise geometric correction (orthorectification) and radiometric normalization. This means the images the authors use for their "on-board" change detection are already perfectly co-registered. In a real on-board scenario, the system would operate on Level-0 (raw) or at best Level-1A/B data, which suffers from:

   • Geometric Misalignment: Jitter, pointing errors, and orbital variations mean that a pixel at coordinate (i,j) in an image from satellite A will not correspond to the same ground location as pixel (i,j) in an image from satellite B without complex on-board orthorectification, which is computationally prohibitive. The authors' change detection would likely be dominated by false positives from this misalignment.
   • Sensor Noise & Radiometric Differences: Different satellites, even with nominally identical sensors, have different noise profiles and spectral response functions. Comparing their raw data directly is non-trivial.
   • The authors acknowledge this in their limitation section (§8, p. 13), but they dismiss it by claiming change detection on low-resolution images is "less sensitive." This is an unsubstantiated claim and does not absolve the work of the need to be validated under realistic conditions. This is not a minor limitation; it is a fundamental threat to the paper's central claim.

2. Oversimplified Illumination Normalization. The paper states that it aligns illumination using "standard linear regression" (§5, p. 8). This is insufficient. Radiometric differences between two satellite passes are highly non-linear due to the Bidirectional Reflectance Distribution Function (BRDF) of ground surfaces. Images taken from different viewing angles and sun angles (which is guaranteed when using different satellites in a constellation) will exhibit significant non-linear intensity changes even if the ground content is identical. A simple linear model cannot correct for this, leading to a high rate of false change detection.

3. Fragility of Change Detection on Heavily Downsampled References. The authors claim they can compress the reference image by over 10,000x (§6, p. 12, Figure 16) by downsampling and still effectively detect changes. Figure 7 (p. 6) claims that a 2600x compression results in only 1.7% of changed tiles being missed. This seems implausible. Downsampling averages out pixel values, making it fundamentally blind to small-scale but high-importance changes (e.g., a new small structure, vehicle movement, initial signs of crop blight). The paper fails to characterize the nature of the missed changes. Are they random noise, or are they systematically the smallest and potentially most valuable new features?

4. Unconvincing Baseline Comparison. The SatRoI baseline is configured as a strawman by using the first available image as a fixed reference for the entire dataset duration (§6.1, p. 9). A far more realistic and stronger baseline would be a "single-satellite best-effort" approach that uses the most recent cloud-free image from the same satellite, even if it is 51 days old on average (as per their own analysis in Figure 5, p. 5). The current comparison conflates the benefit of having a fresh reference with the benefit of having a constellation-based reference, thereby inflating the perceived contribution of Earth+.

5. Underestimation of Uplink Constraints and Operational Risk. The authors dismiss the use of the uplink channel as a non-issue (§8, p. 13), citing low bandwidth usage for control messages. However, this channel is reserved for Telemetry, Tracking, and Command (TT&C) for a reason: it is a mission-critical link. Introducing a routine, high-volume data stream for reference images, however compressed, creates contention and operational risk. A robust system design would require a detailed analysis of TT&C protocols, message prioritization, and the consequences of a failed or delayed reference image update. The paper provides no such analysis.

Questions to Address In Rebuttal

The authors must address the following points to convince me of the validity of their work:

1. Please provide a detailed justification for using pre-processed, co-registered image data for an experiment meant to simulate an on-board processing pipeline. How would your change detection algorithm perform on misaligned Level-0/1A data? At a minimum, please simulate realistic geometric offsets (e.g., sub-pixel and pixel-level shifts) and sensor noise to demonstrate the robustness of your approach.
2. Can you provide evidence that a linear regression model is sufficient for normalizing illumination differences caused by BRDF effects when comparing images from different satellites? Please provide a quantitative analysis of the residual error after your normalization and its impact on the change detection threshold θ.
3. Regarding the 1.7% of changed tiles missed when using a 2600x compressed reference (Figure 7): What is the physical size and nature of the real-world changes that are being missed? Does your method systematically fail to detect changes below a certain spatial scale?
4. Please re-run your evaluation against a stronger "SatRoI-Dynamic" baseline that, for each new image, uses the most recent cloud-free image from that same satellite as its reference. This will properly isolate the benefit of constellation-wide sharing.
5. Provide a more thorough analysis of the proposed use of the TT&C uplink channel. How does your system guarantee that mission-critical command and control functions are not delayed or disrupted by your reference image data stream?

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper presents Earth+, a system for on-board satellite image compression designed to alleviate the critical downlink bottleneck in Earth Observation (EO) constellations. The core contribution is a shift from single-satellite compression paradigms to a constellation-wide, reference-based approach. Instead of each satellite relying on its own, often stale, historical imagery, Earth+ leverages the entire constellation to find the most recent, cloud-free image of a location. This "best" reference image is then downsampled and uploaded to the target satellite via the existing ground-to-satellite uplink. The satellite then captures a new image, compares it to the fresh reference, and only downlinks the geographic tiles that have changed. The authors demonstrate that this system-level architecture can reduce downlink bandwidth usage by up to 3.3x compared to state-of-the-art methods, without sacrificing image quality or exceeding the practical on-board resource constraints of current satellites.

Strengths

The primary strength of this work lies in its elegant and impactful conceptual reframing of the on-board compression problem.

1. A Novel System-Level Architecture: The authors' most significant contribution is not a new compression algorithm, but a new system architecture. By treating the constellation as a cohesive, distributed sensing platform rather than a collection of isolated nodes, they unlock significant efficiency gains. The idea of using the ground segment as a coordination plane to share state (i.e., the best reference image) among satellites is a powerful concept that extends beyond this specific application. It cleverly trades a small amount of low-bandwidth uplink for a large saving on the high-bandwidth downlink, a highly favorable exchange in the resource-scarce environment of space.

2. Pragmatic and Grounded Design: The proposal is firmly rooted in the reality of current EO systems. The authors acknowledge the severe constraints on uplink bandwidth and on-board computation and propose practical solutions. The use of downsampling for the reference image (Section 4.3, pg. 6-7) and incremental updates by caching references on-board are clever techniques that make the system viable. By building upon existing codecs like JPEG-2000 and relying on the existing ground station network, the work presents a plausible pathway to deployment, rather than depending on future technologies like ubiquitous inter-satellite links.

3. Addresses a Critical and Well-Known Problem: The downlink bottleneck is arguably one of the most significant limiters to the value of modern satellite constellations. The paper correctly identifies that over 90% of captured data is discarded (Section 1, pg. 1). A system that can multiply the effective data throughput by a factor of 3.3x is of immense practical importance. This could directly translate to more timely data for critical applications like disaster response, environmental monitoring, and precision agriculture.

4. Strong and Relevant Evaluation: The evaluation is thorough, using two well-chosen datasets: Sentinel-2 to demonstrate performance across diverse geographic content and Planet to show the scaling benefits with a larger number of satellites. The comparison against strong baselines, including Kodan [48] and SatRoI [78], demonstrates a clear and substantial improvement.

Weaknesses

The weaknesses are less about fundamental flaws and more about the practical complexities and un-explored dimensions of this new architecture.

1. Latency and Coordination Complexity of the Ground Segment: The system's effectiveness is predicated on a "ground-in-the-loop" architecture. A reference image from Satellite A must be downlinked, processed on the ground to confirm it is cloud-free, and then scheduled for uplink to Satellite B before B makes its observation pass. The paper glosses over the operational latency and complexity of this pipeline (Section 4.2, pg. 6). This introduces a potential delay that could, in some fast-changing scenarios, reduce the "freshness" of the reference. The architecture also implies a significant data management and coordination burden on the ground stations, which now must serve as a distributed database and content delivery network for the entire constellation.

2. Assumptions of Homogeneity: The evaluation implicitly assumes a relatively homogeneous constellation where an image from one satellite is a suitable reference for another. While the paper mentions handling different bands (Section 5, pg. 8), it does not deeply explore the challenges of using references from satellites with different sensors, ground sampling distances (GSD), viewing angles, or spectral response functions. These cross-sensor inconsistencies are non-trivial and could introduce artifacts that the simple illumination alignment might not correct, potentially polluting the change detection process.

3. Contextual Positioning: While the work is strong, the authors could better position it within broader computing trends. Earth+ is an excellent example of "distributed systems in space" and a specialized form of edge computing. Explicitly framing the work in this context would highlight its relevance beyond the remote sensing community, connecting it to ongoing research in federated systems and resource management at the extreme edge.

Questions to Address In Rebuttal

1. Operational Timeline: Could the authors elaborate on the practical, end-to-end latency of the reference image pipeline? Specifically, what is the expected time from the moment Satellite A captures a potential reference image to the moment that reference is successfully uploaded and available on Satellite B? How does this latency impact the choice of which reference to upload, especially if a slightly older but already-downlinked image is available versus a fresher one still on-board another satellite?

2. Robustness to Heterogeneity: How sensitive is the change detection mechanism to variations between sensors in a heterogeneous constellation? For instance, if the reference image is from a sensor with a 4.1m GSD and the new image is from one with a 3.0m GSD (as in the Planet dataset, Table 2), how is this difference in resolution handled before the pixel-wise comparison? Does the system's effectiveness degrade as the sensor characteristics diverge?

3. Graceful Degradation: The system's main benefit relies on a functional uplink. How does Earth+ degrade in the face of intermittent or unavailable uplink connectivity to a given satellite? The paper mentions caching old references (Section 4.3, pg. 7), but could the authors quantify the trade-off? For instance, how much does the downlink saving decrease for each day the reference image is not updated? Is there an automated fallback to a non-reference mode?

Reviewer: The Innovator (Novelty Specialist)

Summary

The authors present Earth+, a system for on-board satellite imagery compression. The core idea is to improve upon existing reference-based encoding schemes by addressing the problem of stale, cloud-covered reference images. The central claim of novelty lies in its system-level architecture: instead of relying on historical images from a single satellite, Earth+ leverages the entire satellite constellation. Images from all satellites are downlinked to ground stations, where the freshest, most cloud-free image of a given location is selected as a reference. This optimal reference is then uplinked to the next satellite scheduled to pass over that location. The satellite uses this fresh reference to perform change detection, encoding and downlinking only the tiles that have changed. The authors claim this constellation-wide approach significantly reduces the age of reference images, thereby decreasing the amount of "changed" area that must be downlinked, achieving up to a 3.3x reduction in downlink usage compared to state-of-the-art methods.

Strengths

The primary strength of this paper is its architectural novelty. My analysis of prior art confirms that the core contribution—the specific data flow of using the ground segment as a central broker to source reference images from an entire constellation and uplink them to a target satellite—is genuinely new in the context of satellite image compression.

1. Novel Inversion of Data Flow: The canonical view of satellite communication is data flowing down. The use of the scarce uplink channel to send reference data up to enable better compression on the downlink is a clever and non-obvious system design. It re-frames the uplink from a pure command-and-control channel into an active component of the compression pipeline.

2. Significant Delta Over Prior Art: The paper correctly identifies and cites the most relevant prior art in on-board reference-based compression, namely SatRoI [78] and related works [88]. These systems are fundamentally single-satellite solutions, limited by the age and quality of images they have captured themselves. The "delta" introduced by Earth+ is the entire constellation-wide sharing mechanism. This is not an incremental algorithmic tweak but a paradigm shift in how reference frames can be sourced, moving from a local, historical cache to a global, near-real-time selection pool.

3. Feasibility of the Novel Concept: A novel idea is only as good as its practicality. The authors astutely identify that the limited uplink bandwidth is the Achilles' heel of their proposal. Their solution—aggressively downsampling the reference image and only uploading changes to a cached on-board reference (Section 4.3, page 6-7)—is a critical and novel enabling technique. The finding, shown in Figure 7 (page 6), that a reference image can be compressed >2600x and still be effective for change detection, provides strong evidence that this architecture is not merely theoretical but practically viable within existing constraints.

Weaknesses

While the system architecture is novel, the work's contribution is almost entirely dependent on this single idea. The underlying components are not new, and the novelty may be contingent on a specific technological state of satellite networking.

1. Component-Level Unoriginality: The paper's novelty is purely architectural. The individual building blocks—change detection via thresholded pixel differences, the use of JPEG-2000, and decision-tree-based cloud detection—are all well-established, standard techniques. The contribution rests solely on the innovative way these existing components are connected. Should a fundamental flaw in the proposed architecture be found, the paper offers little in the way of other novel contributions.

2. Contingent Novelty: The entire premise of using the ground as a relay is predicated on the absence of high-bandwidth, low-latency inter-satellite links (ISLs). The authors acknowledge this in Section 4.2 (page 6), dismissing ISLs as "not currently available for earth observation satellites". While true for many current systems, this is a rapidly evolving area. If constellations with ubiquitous ISLs (akin to Starlink) become the norm for earth observation, the ground-based relay proposed here could be rendered obsolete by a more efficient in-orbit reference sharing protocol. The novelty of the solution is therefore tied to, and potentially limited by, the current state of satellite hardware.

3. Concept Overlap with Video Coding Principles: The technique described as "Incrementally updating reference images" in Section 4.3 (page 7) is conceptually very similar to the use of I-frames (full reference) and P/B-frames (delta encoding) in standard video codecs. The authors patch an on-board reference with uplinked changes. While the source of these changes (ground-selected) is novel, the mechanism of patching a reference frame is functionally analogous to long-standing principles in video compression. The paper could do a better job of distinguishing its incremental update mechanism from this vast body of prior art.

Questions to Address In Rebuttal

1. The core idea involves a central ground station sending updated reference models (images) to distributed edge sensors (satellites) to reduce their data transmission. Can the authors comment on whether this architectural pattern has precedents in other, non-satellite domains (e.g., terrestrial IoT, distributed robotics) and clarify how their contribution is novel with respect to that potential body of work?

2. Please elaborate on the novelty of the proposed system in a future where high-bandwidth ISLs are commonplace. Would the Earth+ system become obsolete, or could its core logic (constellation-wide selection of a reference) be adapted to an in-orbit protocol? If so, would such an adaptation be a trivial extension of this work or a significant research contribution in its own right?

3. The proposed system relies on a threshold (θ) for change detection. How does the severe downsampling of the reference image affect the selection and robustness of this threshold? Is a single, globally-profiled θ (as mentioned in Section 5, page 8) sufficiently novel and robust, or does it ignore the complexities introduced by the novel compression scheme itself, such as aliasing and loss of fine-grained textures?

Paper: EDM: An Ultra-Low Latency Ethernet Fabric for Memory Disaggregation Reviewer: The Guardian

---

Summary

This paper proposes EDM, a radical redesign of the Ethernet fabric for memory disaggregation that aims to achieve ultra-low latency. The core proposal involves two aggressive architectural changes: 1) moving the entire network protocol stack for remote memory access from above the MAC layer into the Physical (PHY) layer, and 2) implementing a centralized, priority-based PIM scheduler within the switch's PHY layer to create virtual circuits and eliminate queuing. The authors support their claims with a small-scale 25GbE FPGA hardware prototype, which demonstrates a ~300ns unloaded latency, and larger-scale C-language simulations to evaluate performance under load. While the stated performance goals are ambitious, the work rests on a series of optimistic assumptions and its evaluation lacks the scale and rigor to substantiate its claims of practical viability.

Strengths

1. Problem Motivation: The paper does an excellent job of identifying and articulating the fundamental latency and bandwidth overheads imposed by the standard Ethernet stack (MAC layer constraints, IFG, L2 switching latency) for small, latency-sensitive memory messages (Section 2.4, page 4). This background provides a clear and compelling motivation for exploring unconventional solutions.
2. Scheduler Design: The design of the priority-based PIM scheduler in hardware is detailed and appears well-considered (Section 3.1.2, page 7). The use of constant-time data structures for priority queue operations and the 3-cycle implementation of a PIM iteration demonstrate a clear path toward a high-performance ASIC implementation.
3. Holistic Approach: The authors present an end-to-end design, considering the necessary modifications at the host NIC, the switch, and the protocol semantics. This is a commendable effort compared to papers that focus only on one piece of the puzzle.

Weaknesses

My primary concerns with this work revolve around its fundamental feasibility, the limited scope of its evaluation, and the dismissal of significant real-world complexities.

1. Fundamental Violation of Standardization: The core premise of implementing a custom protocol within the PHY layer (Section 3.2, page 8) is a critical flaw from a practical standpoint. The PHY layer is a complex and highly standardized domain for a reason. The paper completely fails to address how EDM interacts with essential PHY-level functions like Forward Error Correction (FEC), Auto-Negotiation, or training sequences. These are not trivial features; they are essential for reliable communication over modern high-speed links. By creating custom /M* block types, the authors effectively propose a proprietary, non-standard physical layer that would be incompatible with the entire existing ecosystem of transceivers, optics, and diagnostic tools. The claim in Section 3.3 (page 10) that EDM simply "creates a parallel pipeline" is a gross oversimplification that ignores the physical and logical realities of the SerDes interface.
2. Insufficient and Unrepresentative Hardware Evaluation: The hardware results, while showing impressive latency, are derived from a toy-scale testbed consisting of just two host nodes and one 2-port switch (Section 4.2, page 11). Extrapolating performance from this minimal setup to a 512-port switch, as is implied by the ASIC synthesis discussion (Section 4.1), is a significant logical leap. Such a small system cannot exhibit the complex contention patterns, scheduler hotspots, or clock-domain crossing challenges that emerge at scale. The ~300ns latency figure is an ideal, best-case number that has not been validated under any meaningful stress.
3. Understated Overheads and Unaddressed Trade-offs:
   • Write Latency Penalty: The design requires an explicit notification for write requests (WREQ), incurring an RTT/2 latency penalty before data can be sent (Section 3.1.1, page 6). The authors dismiss this as "nominal" and a "small price to pay" (Section 3.1.4, page 8). This is not a credible assessment. In a rack- or cluster-scale network, the propagation delay alone can be hundreds of nanoseconds to several microseconds, making this "small price" potentially larger than the entire unloaded latency EDM claims to achieve. This fundamentally biases the fabric's performance towards read-heavy workloads, a point that is not adequately analyzed.
   • Penalty on Standard Traffic: The intra-frame preemption mechanism (Section 3.2.3, page 10) imposes a buffering delay on all non-memory traffic on the receive path, equal to the transmission delay of a maximum-sized frame. For a 9KB jumbo frame on a 100Gbps link, this is a ~720ns latency tax paid by all standard IP and storage traffic, even when there are no memory messages to preempt. This is a significant performance regression for co-existing traffic that the paper fails to evaluate or even acknowledge.
4. Grossly Simplified Handling of System-Level Challenges: The section on "Practical Concerns" (Section 3.3, page 10) hand-waves away monumental engineering challenges. The proposed solution for fault tolerance—state machine replication for the in-switch scheduler state—is presented as a straightforward extension. In reality, implementing state replication for a nanosecond-scale, line-rate scheduler without compromising performance is an enormous, unsolved research problem that would introduce significant latency and complexity. Dismissing this with a single paragraph is a serious omission.
5. Questionable Simulation Baselines: The simulation results that show CXL performing up to 8x worse than EDM (Figure 8b, page 14) are highly suspect. The paper attributes this to credit-based flow control and head-of-line blocking (Section 4.3.1, page 13). While this is a known potential issue, an 8x performance degradation suggests that the CXL model used in the simulation may be uncharitable or configured to exhibit a worst-case pathology that is not representative of typical CXL switch implementations or workloads. Without a detailed validation of the CXL simulation model, these comparative results lack credibility.

Questions to Address In Rebuttal

1. Please provide a detailed account of how EDM's PHY-layer modifications would co-exist with standard and essential PHY functions like FEC (e.g., Reed-Solomon RS-FEC), link training, and auto-negotiation. How can a standard transceiver correctly interpret a link that interleaves standard 66-bit blocks with EDM's custom /M* blocks?
2. Can the authors justify the claim that the performance observed in a 2-node, 1-switch FPGA testbed is representative of a large-scale deployment? What scheduler or system-level bottlenecks do you anticipate when scaling from 2 ports to 128 or 512 ports, and why are these not captured in your evaluation?
3. Provide a quantitative analysis of the RTT/2 latency overhead for write operations. At what physical distance (e.g., 10m, 50m, 100m optical links) does this overhead cease to be "nominal" and instead dominates the total transaction latency?
4. Please provide a detailed specification of the CXL model used in your C-language simulator. What are the specific buffer sizes, credit exchange parameters, and traffic patterns that lead to the 8x worse message completion time shown in Figure 8b? Please provide evidence that your model accurately reflects the behavior of state-of-the-art CXL fabrics.
5. Regarding fault tolerance, the paper proposes state machine replication. Can you elaborate on how you would implement this for a scheduler that must make decisions every few nanoseconds without introducing significant latency that would nullify EDM's primary benefit? Please acknowledge the state-of-the-art in this area and how your proposal relates to it.

Reviewer Persona: The Synthesizer (Contextual Analyst)

---

Summary

This paper presents EDM, a novel network fabric designed to achieve ultra-low latency for memory disaggregation over Ethernet. The authors identify the standard Ethernet protocol stack, particularly the MAC layer, as a fundamental latency and overhead bottleneck for small, latency-sensitive memory messages.

The core contribution is a radical architectural shift: bypassing the MAC layer entirely and implementing a specialized network protocol for remote memory access directly within the Ethernet Physical Layer (PHY). This is complemented by a second key idea: a fast, centralized, in-network scheduler, also implemented in the switch's PHY. This scheduler creates dynamic, nanosecond-scale virtual circuits between compute and memory nodes, eliminating L2 processing and queuing delays.

Through an FPGA-based prototype and larger-scale simulations, the authors demonstrate that EDM can achieve a remote memory access latency of ~300 ns, which is an order of magnitude better than existing Ethernet-based RDMA solutions and is competitive with emerging PCIe-based fabrics like CXL, while retaining the scalability and cost benefits of Ethernet.

---

Strengths

The true strength of this paper lies in its ambition and the elegance of its core idea. It challenges a long-held assumption about network layering and, in doing so, opens a compelling new design point for datacenter fabrics.

1. A Bold and Foundational Contribution: The central idea of moving the memory fabric into the PHY is not an incremental optimization; it is a fundamental re-architecting of the network stack for a specific, high-value workload. By identifying the MAC layer's frame-based semantics (minimum frame size, inter-frame gap, lack of preemption) as the root cause of latency for small messages (Section 2.4, page 4), the authors make a convincing case for their radical approach. This is the kind of high-level conceptual thinking that can inspire a new line of research.

2. Excellent Contextualization and Positioning: The authors do a superb job of placing their work within the current landscape of memory disaggregation. The comparison is not just against other Ethernet protocols but squarely against CXL, the leading alternative fabric. The paper effectively frames EDM as a way to achieve CXL-like latency without abandoning the cost, distance, and bandwidth-scaling advantages of the Ethernet ecosystem (Section 2.2, page 3). This demonstrates a keen awareness of the broader industry and academic trends.

3. A Holistic and Well-Considered System Design: The work goes far beyond a single clever trick. The design is comprehensive, encompassing the host NIC, the switch, and the scheduling protocol that ties them together. The in-PHY scheduler, inspired by Parallel Iterative Matching (PIM), is thoughtfully designed to be implemented in high-speed hardware (Section 3.1.2, page 7). Furthermore, the design for intra-frame preemption (Section 3.2.3, page 10) is a crucial and elegant feature that acknowledges the reality of converged networks where memory traffic must coexist with traditional IP traffic. This demonstrates deep systems thinking.

4. Connecting Disparate Concepts: This work synthesizes ideas from several domains. It takes the classic networking concept of virtual circuits, implements it using modern high-speed scheduling algorithms (PIM), and places it in a novel part of the network stack (the PHY), a layer previously explored more for timing or covert channels. By applying PHY-level engineering to a mainstream datacenter problem, the paper connects what were once niche research areas to a problem of significant practical importance.

---

Weaknesses

While the core idea is powerful, the paper would be strengthened by a more direct engagement with the practical and systemic challenges its adoption would entail. My concerns are less about the validity of the idea and more about its path to real-world impact.

1. The "Ecosystem" Barrier to Adoption: The most significant challenge for EDM is that it requires a non-standard PHY in both the NIC and the switch. This creates a chicken-and-egg problem for adoption. Unlike software-based solutions or even P4-based programmable switches that work within the existing Ethernet framework, EDM requires new hardware from the ground up. This is a formidable barrier and represents the biggest threat to the work's practical impact.

2. The Limits of Centralized Scheduling: The proposed design is centered on a single, centralized scheduler within one switch, targeting rack- or cluster-scale deployments. This is a reasonable starting point, but the broader vision for datacenter-scale memory disaggregation will inevitably involve multi-switch topologies. The paper does not discuss how the EDM model would extend beyond a single switch domain. Would traffic revert to standard RoCEv2 when crossing switch boundaries, negating the latency benefits? A discussion of the architectural implications for larger-scale networks is a missing piece of the puzzle.

3. Understated Engineering Complexity in ASICs: While the authors have successfully prototyped their design on FPGAs using an open-source PHY, translating this to a commercial, multi-terabit switch ASIC is a non-trivial leap. The PHY/SerDes in modern ASICs is a highly complex piece of mixed-signal hardware, often licensed as hardened IP. The paper could better acknowledge the engineering challenges of modifying this layer, especially concerning signal integrity, clocking domains, and integration with existing FEC (Forward Error Correction) logic, which also operates at the PHY level.

---

Questions to Address In Rebuttal

1. Path to Impact: Given the requirement for custom hardware on both the host and switch side, what do you envision as a plausible adoption path for EDM? Could a version of this be implemented using emerging technologies like programmable SmartNICs and programmable switches, even if at a higher latency, as a bridge to full ASIC integration?

2. Scaling Beyond a Single Switch: Could you elaborate on how you see the EDM architecture evolving for multi-rack or datacenter-scale deployments? How would an EDM-enabled rack interconnect with another, and what would the end-to-end latency properties of such a connection be? Does the centralized scheduling model fundamentally limit EDM to a single-switch domain?

3. ASIC Implementation Feasibility: Based on your work, what are the most critical interactions between EDM's logic and the core functionality of a modern high-speed Ethernet SerDes (e.g., equalization, FEC)? Do you believe that EDM's logic can be cleanly separated from the analog and signal-processing components of the PHY, making it a feasible addition to future switch/NIC ASICs?

Reviewer: The Innovator (Novelty Specialist)

Summary

The paper presents EDM, an ultra-low latency Ethernet fabric designed for memory disaggregation. The core proposal is to circumvent the traditional network stack by implementing a complete protocol for remote memory access, including a centralized in-network scheduler, entirely within the Physical (PHY) layer of the Ethernet NIC and switch. By operating at the granularity of 66-bit PHY blocks, the authors claim to eliminate the fundamental overheads of the MAC layer (minimum frame size, inter-frame gap, lack of preemption) and the processing latency of the switch's L2 forwarding pipeline. The authors demonstrate a ~300ns end-to-end fabric latency on an FPGA prototype, an order of magnitude lower than standard Ethernet-based protocols.

Strengths

The primary strength of this paper lies in the novelty of its architectural design point. While individual components of the proposed system have conceptual precedents in prior work, their synthesis into a cohesive, high-performance system operating exclusively at the PHY layer is, to my knowledge, new. The specific novel contributions are:

1. Protocol Relocation: The radical proposition of moving the entire network protocol stack for a specific traffic class (memory access) into the PHY is the paper's most significant novel claim. This is a fundamental departure from decades of layered network architecture.
2. Scheduler Integration: Placing a centralized, hardware-accelerated scheduler inside the switch PHY (Section 3.1, page 5) is a clever and novel mechanism. This placement is the key enabler that allows EDM to create virtual circuits and bypass the entire L2 packet forwarding pipeline, which is a major source of latency in conventional switches.
3. PHY-level Preemption: The mechanism for intra-frame preemption at the 66-bit block level (Section 3.2.3, page 10) is a novel implementation. It provides a much finer granularity of control than existing MAC-level preemption standards (e.g., IEEE 802.3br), which is critical for protecting latency-sensitive memory traffic.

The performance benefits demonstrated are not marginal; a reduction in fabric latency by an order of magnitude is substantial enough to justify the consideration of such a non-standard and complex architecture.

Weaknesses

While the overall architecture is novel, the paper could do a better job of positioning its contributions against the closest conceptual prior art. The novelty is in the synthesis and location, not necessarily in the foundational concepts themselves.

1. Repurposing PHY Constructs: The idea of using idle or otherwise unused portions of the PHY layer for data transmission is not entirely new. Prior work on PHY-level covert channels, such as Lee et al. [37] ("PHY Covert Channels: Can you see the Idles?"), established the principle of repurposing idle characters for out-of-band communication. The authors cite this work but should more explicitly frame their contribution as elevating this concept from a low-bandwidth "covert channel" to a first-class, high-bandwidth protocol, which is a significant delta but builds on the same foundational insight.
2. Scheduling Algorithm: The core scheduling algorithm is a priority-based version of the classic Parallel Iterative Matching (PIM) from Anderson et al. [6]. The novelty here is not the algorithm but its highly optimized, constant-time hardware pipeline implementation (Section 3.1.2, page 7) and its unique placement. The paper should be more precise in claiming novelty for the implementation and integration, rather than the scheduling algorithm itself.
3. Centralized Scheduling: The concept of a centralized scheduler for datacenter networks was explored in depth by works like Fastpass [51]. Fastpass, however, used a separate server, which introduced different bottlenecks. EDM's novelty is in decentralizing the scheduler logic to the switch hardware itself, specifically the PHY. The paper should more clearly articulate this distinction: it is an in-network centralized scheduler, not a server-based one.

Questions to Address In Rebuttal

The authors should use the rebuttal to sharpen the articulation of their novel contributions.

1. Novelty vs. PHY Covert Channels: How do the authors differentiate their work from the conceptual precedent set by papers like [37], which proposed repurposing PHY-level constructs for data transfer? Is the primary novelty the scale (a full protocol vs. a covert channel) or a more fundamental architectural difference? Please clarify the key inventive step that allows for this scaling.
2. Novelty of the Scheduler: The paper's scheduler is an optimized hardware implementation of the well-known PIM algorithm. Could the authors confirm if there are any novel algorithmic contributions to the scheduler itself, or if the novelty lies exclusively in its efficient hardware pipeline and its unique placement within the PHY?
3. Comparison to Preemption Standards: The IEEE 802.3br standard defines frame preemption at the MAC layer to support time-sensitive networking. While the authors' PHY-level approach is different and finer-grained, could they comment on the trade-offs and explain why the existing standard is insufficient for their goals? This would help solidify the necessity of their novel approach.
4. Justifying Complexity: The proposed architecture requires deep, non-standard modifications to the Ethernet PHY in both NICs and switches, a significant engineering and ecosystem cost. Given that CXL offers comparable unloaded latency, and the primary benefit of EDM appears under heavy network load (Figure 8, page 14), what specific class of applications justifies this massive trade-off of standardization for performance? Is the primary driver truly memory disaggregation, or is it a more general fabric for HPC/ML workloads where congestion is the dominant problem?

Reviewer Persona: The Guardian (Adversarial Skeptic)

---

Summary

The authors present "Plaid," a novel Coarse-Grained Reconfigurable Array (CGRA) architecture and an accompanying compiler framework. The central thesis is that conventional CGRAs overprovision communication resources relative to their compute capabilities. To address this, Plaid introduces a hierarchical execution model based on "motifs"—small, recurring 3-node dataflow patterns. The architecture features Plaid Collective Units (PCUs), each designed to execute a single motif collectively using a local router, with multiple PCUs connected via a global mesh network. The authors claim that this approach significantly reduces power (43%) and area (46%) compared to a spatio-temporal CGRA baseline, while preserving performance and generality.

Strengths

1. Interesting Premise: The core intuition that the generalized, fine-grained connectivity in many CGRAs may be inefficient for common, localized dataflow patterns is plausible and presents an interesting direction for architectural optimization.
2. End-to-End System: The authors have undertaken the commendable effort of co-designing both a hardware architecture and a complete compiler toolchain to support their proposed execution model. This provides a comprehensive view of the system's potential.
3. Clear Architectural Concept: The proposed hierarchical architecture, with a clear separation between local (intra-motif) and global (inter-motif) communication, is a logical and well-articulated design that directly follows from the paper's central premise.

Weaknesses

My primary concerns with this submission relate to the unsubstantiated foundational claims, the questionable rigor of the experimental evaluation, and contradictions between the stated goals and the presented results.

1. The "Motif" Concept is Insufficiently Justified: The entire architecture is built upon the primacy of the 3-node motif. However, the justification for N=3 (Section 3.2, page 5) is anecdotal rather than empirical. The authors claim larger motifs are rare and smaller ones are trivial, but they provide no supporting data from a broad analysis of application DFGs. The argument that the three chosen motifs are "exhaustive, fundamental building blocks" is presented without a formal proof and appears to be an oversimplification. The choice of N=3 feels convenient for the proposed hardware, not a conclusion drawn from rigorous data analysis.

2. The "Generality" Claim is Contradicted by Evidence: The paper repeatedly claims to preserve the generality of CGRAs. However, the performance results in Figure 12 (page 10) and the subsequent discussion on page 11 directly undermine this. The authors concede that Plaid's performance degrades on kernels such as atax_u4 and seidel_u2 due to "more complex and long data dependencies." This is a critical admission: the architecture is demonstrably not as general as the baseline and is biased towards applications that decompose neatly into the predefined local motifs. An architecture that falters on specific, valid communication patterns cannot be considered fully general-purpose.

3. Baselines are Vague and Unverifiable: The experimental comparison, and thus the paper's headline results, rests on poorly defined baselines (Section 6.3, page 9).

   • The primary "spatio-temporal CGRA" baseline is described merely as "typical" with a 4x4 mesh. This is not a reproducible scientific standard. Which published architecture's router, buffer depth, and connectivity does it model? Without these specifics, the reported 43% power and 46% area savings are meaningless, as they could easily be the result of comparing Plaid against a non-optimized or "strawman" baseline.
   • The "spatial CGRA" baseline relies on a custom Python script for DFG partitioning. The performance of spatial architectures is notoriously sensitive to the quality of this partitioning. The authors provide no validation of their script's efficacy, leaving open the possibility that this baseline was artificially handicapped.

4. Impact of Incomplete Motif Coverage is Ignored: The authors' own data in Table 2 (page 9) shows that for several key benchmarks, a substantial portion of compute nodes are not covered by motifs (e.g., dwconv_u5 has 13 of 19 compute nodes covered, leaving ~32% as "standalone"). These standalone nodes must use the global network, presumably negating the core architectural benefit of localized routing. The paper fails to analyze the performance and energy overhead incurred by this non-trivial fraction of the workload, which represents a significant hole in the evaluation.

Questions to Address In Rebuttal

The authors must address the following points to substantiate the claims made in this paper:

1. Provide a rigorous, data-driven justification for selecting the 3-node motif as the fundamental architectural primitive. This must include a statistical analysis of N-node motif prevalence and complexity across a wide and diverse set of benchmarks, demonstrating that N=3 is indeed an optimal design point.

2. Provide a precise, detailed, and reproducible specification of the "spatio-temporal CGRA" baseline architecture. Which specific, published CGRA design does it model? Specify the router microarchitecture, number of virtual channels, buffer sizes, and crossbar implementation used for the power and area analysis.

3. The performance degradation on certain kernels is a key finding. Please provide a deeper, quantitative analysis of the "complex data dependencies" that cause this slowdown. How does this finding modify your central claim of preserving generality? Is there a class of algorithms for which Plaid is fundamentally unsuited?

4. Quantify the performance and energy impact of the "standalone" nodes that are not covered by motifs. For a benchmark like dwconv_u5, where nearly a third of compute nodes are standalone, how much of the overall execution time and energy is spent on these less-efficient operations and their corresponding global communication?

Review Form: The Synthesizer (Contextual Analyst)

---

Summary

This paper introduces Plaid, a novel Coarse-Grained Reconfigurable Array (CGRA) architecture and compiler co-design aimed at resolving the well-known problem of communication resource overprovisioning in traditional CGRA designs. The authors' core contribution is the insight that dataflow graphs (DFGs) are not random but contain recurring, simple communication patterns, which they term "motifs."

Instead of equipping every processing element (PE) with a powerful, and thus costly, router, Plaid introduces a hierarchical execution model. The architecture is built from Plaid Collective Units (PCUs), where each PCU contains multiple ALUs and a lightweight "local router" designed to efficiently handle the internal communication of these motifs (specifically 3-node patterns like fan-in, fan-out, etc.). A higher-level "global router" network then manages the more complex, long-distance communication between these PCUs. This architectural concept is tightly coupled with a compiler that can automatically identify these motifs within a DFG and map them hierarchically onto the Plaid fabric. The results demonstrate significant improvements in power (43% reduction) and area (46% reduction) compared to a conventional high-performance spatio-temporal CGRA, while maintaining comparable performance and generality.

Strengths

1. Elegant Solution to a Fundamental Problem: The paper astutely identifies and addresses a fundamental tension in CGRA design: the high cost of providing full communication flexibility. The observation that communication is often locally structured is insightful, and the proposed solution—a hierarchical network tailored to these structural motifs—is both elegant and effective. This moves the field beyond simply making incremental improvements to existing flat PE array architectures.

2. Novel and Principled Architectural Abstraction: The concept of "collective execution" within a PCU represents a powerful new architectural abstraction. It effectively creates a higher-level instruction set for the CGRA, where an "instruction" can be an entire communication motif rather than a single ALU operation. By formalizing this around the exhaustive set of 3-node DAGs (Section 3.2, page 5), the authors provide a principled foundation for their design, striking a compelling balance between specialization (for motifs) and generality (retaining reconfigurability).

3. Strong Co-design and System-Level View: The success of this work lies in its tight integration of hardware and software. The Plaid architecture would be ineffective without a compiler capable of exploiting it. The authors present a complete toolchain, including algorithms for motif identification and hierarchical mapping (Section 5.2, page 8), demonstrating a mature, system-level perspective that is crucial for practical accelerator design.

4. Context and Significance: This work fits beautifully within the broader trend of domain-specific and specialized computing. While many approaches focus on specializing compute units (e.g., hardwiring operator chains), Plaid focuses on specializing the communication fabric in a structured, reconfigurable way. This is a novel perspective that could influence not only future CGRAs but also other dataflow-style accelerators, providing a systematic framework for aligning compute and communication resources rather than relying on ad-hoc specializations.

Weaknesses

While the core idea is strong, the work could be strengthened by addressing the following points:

1. Assumption of Motif Dominance: The entire premise rests on the idea that DFGs can be effectively decomposed into 3-node motifs. While this appears true for the benchmarks evaluated, the paper does not fully explore the architecture's sensitivity to DFG structure. The performance on applications dominated by sparse, irregular, or long-range dependencies, which may not decompose cleanly into local motifs, is unclear. The work would be more robust with a sensitivity analysis or discussion on the architectural "cliff" for non-motif-friendly workloads.

2. Compiler Heuristics: The motif generation algorithm relies on an iterative, randomized process of breaking and regenerating motifs (Algorithm 1, page 8). This is a reasonable heuristic, but its performance and convergence properties are not characterized. For very large and complex DFGs, this process could become a bottleneck or converge to suboptimal solutions, impacting the overall quality of results.

3. Global Network Scalability: The paper demonstrates scalability from a 2x2 to a 3x3 PCU array (Section 7.2, page 12), but the analysis of the global network as a potential bottleneck at larger scales is limited. As the array size increases, the latency and contention on this shared "conveyor belt" will inevitably become more significant. A more detailed analysis of the trade-offs and pressure on the global interconnect would provide a clearer picture of Plaid's scalability limits.

Questions to Address In Rebuttal

1. The foundation of Plaid is the 3-node motif. How would the architecture and its performance adapt to applications where larger, more complex subgraphs (e.g., 4- or 5-node patterns) are the dominant computational structures? Is the framework extensible to identify and collectively route larger motifs, and what would be the architectural implications?

2. Could you provide more insight into the behavior of the motif generation compiler pass (Algorithm 1)? Specifically, how does the number of motifs generated and the overall quality of the hierarchical DFG improve over the iterations compared to the initial greedy approach? What is the typical compile-time overhead of this iterative refinement?

3. Could you elaborate on the latency and resource trade-offs between the local and global networks? For a critical path in a DFG that spans multiple motifs mapped to non-adjacent PCUs, how does the multi-hop global network traversal time impact the achievable Initiation Interval (II) compared to a flat network where PEs might be placed closer together?

Reviewer: The Innovator (Novelty Specialist)

Summary

This paper introduces "Plaid," a novel CGRA architecture and accompanying compiler framework. The central claim of novelty rests on a paradigm of "hierarchical execution" which is enabled by two core ideas. First, the authors propose that arbitrary dataflow graphs (DFGs) can be systematically decomposed into a small, exhaustive set of recurring 3-node communication patterns, which they term "motifs" (fan-in, fan-out, unicast). Second, they present a co-designed hardware unit, the Plaid Collective Unit (PCU), specifically architected to execute these motifs collectively using a local router. These PCUs are then interconnected via a global network, creating a hierarchical on-chip network. The authors claim this alignment of compute and communication provisioning significantly reduces the power and area overhead typical of spatio-temporal CGRAs without sacrificing performance or generality.

Strengths

The primary strength of this paper lies in the conceptual elegance of its core idea. The attempt to formalize the fundamental building blocks of dataflow beyond the single-node level into a small, exhaustive set of "motifs" is a compelling and novel approach to the CGRA design problem. Instead of relying on ad-hoc pattern identification from specific application domains, the authors derive their 3-node motifs from first principles of graph theory, lending the approach a strong sense of generality.

The tight co-design of the architecture (the PCU) and the compiler is another significant strength. The PCU is not an arbitrary cluster of PEs; it is a direct physical manifestation of the motif concept. This demonstrates a holistic design philosophy that is often missing in proposals that focus on either hardware or software in isolation. The resulting system appears to strike a new and potentially valuable trade-off point between the flexibility of traditional spatio-temporal CGRAs and the efficiency of more specialized or spatial designs.

Weaknesses

While the proposed system is well-conceived, its core conceptual novelty is not as profound as presented when viewed against the full landscape of prior art. The central idea of identifying and accelerating common subgraphs or dataflow patterns is not new. The authors themselves briefly acknowledge CCA [4, 5] in Section 8 (page 13), which proposed accelerating "commonly observed dataflow semantics" with "composable rows of functional units." While the authors argue Plaid's network is more flexible, the fundamental premise of grouping operations for collective execution is conceptually overlapping. The paper would be significantly stronger if it dedicated more space to a direct and detailed comparison with CCA, moving beyond the brief mention in the related work to explicitly articulate the delta in terms of motif derivation (systematic vs. empirical) and hardware implementation (reconfigurable local router vs. composable rows).

Furthermore, the claim that a 3-node motif is the optimal, fundamental unit feels more like a well-reasoned assertion than an empirically proven fact. The justification in Section 3.2 (page 5) is logical, but the paper lacks a sensitivity analysis. What is the impact of kernels that are dominated by 4-node or 5-node patterns? How much overhead is incurred by decomposing these into 3-node motifs and standalone nodes? The novelty of the solution is tied to its effectiveness, and its effectiveness on "non-ideal" DFGs is not thoroughly explored.

Finally, the claim of novelty for the "hierarchical on-chip network within a single CGRA" (Section 3, page 3) needs more rigorous defense. While hierarchical networks are common in many-core processors, their specific application and novelty in the CGRA context should be more clearly substantiated against prior CGRA interconnect designs.

Questions to Address In Rebuttal

1. Regarding Prior Art (CCA): Please provide a more detailed and quantitative comparison to the subgraph acceleration proposed in CCA [4, 5]. Beyond a qualitative statement on flexibility, how does your systematic motif derivation differ from their approach? Could the Plaid architecture be considered a more generalized and reconfigurable evolution of the core concept presented in CCA, and if so, what is the key inventive step that separates it?

2. On the Universality of the 3-node Motif: The paper's foundation rests on the primacy of the 3-node motif. Can you provide data on the distribution of motif sizes in your benchmarks? What percentage of DFG nodes are "left over" as standalone nodes after the motif generation process in Algorithm 1? How does the performance and efficiency of Plaid degrade for kernels that are structurally resistant to decomposition into 3-node patterns?

3. On Architectural Novelty: Could you please elaborate on the novelty of the hierarchical NoC specifically within the context of prior CGRA architectures? What are the closest preceding interconnect designs in the CGRA literature, and what makes the two-level local/global routing in Plaid a fundamentally new approach for this class of accelerators?

Paper Title: Exo 2: Growing a Scheduling Language Reviewer Persona: The Guardian (Adversarial Skeptic)

---

Summary

This paper presents Exo 2, a user-schedulable language (USL) designed to be "growable," meaning users can define new scheduling operations and abstractions externally to the compiler. The authors identify three pillars for such a system: actions (transformations), inspection (code queries), and references (pointing to code). They introduce a mechanism called "Cursors" to unify these concepts, aiming to provide stable, relative references that persist across code transformations. The core claim is that this design allows for the creation of powerful, reusable scheduling libraries, which they demonstrate by building libraries for BLAS and image processing. They claim this approach reduces scheduling code by an order of magnitude and delivers performance competitive with state-of-the-art, hand-optimized libraries.

Strengths

Despite my significant reservations, the paper does possess some merits that warrant acknowledgment.

1. A Coherent Conceptual Framework for References: The paper’s primary strength is the taxonomy of reference mechanisms in USLs presented in Section 5. The categorization along the axes of nominal vs. relative, single vs. multiple, and stable vs. one-time is a valuable conceptual contribution. It provides a clear lens through which to analyze and compare the design of any transformative meta-programming system.

2. Demonstrated Expressiveness: The authors successfully demonstrate the expressiveness of their system by re-implementing complex scheduling operations from other languages. The user-space implementation of a Halide-like compute_at (Section 6.3.2), which requires non-trivial inspection (bounds inference) and action, is a convincing proof of concept.

3. Emphasis on Inspection: The paper correctly identifies the lack of robust inspection capabilities as a key limitation in existing USLs (Section 4). Elevating inspection to a first-class primitive is a necessary step towards enabling more intelligent and automated scheduling abstractions, and the example of user-defined bounds inference (Section 5, page 5) is compelling.

Weaknesses

The paper's claims rest on a foundation that, upon close inspection, appears critically flawed. The work suffers from unsubstantiated claims, a fundamental lack of rigor in its core mechanism, and a misattribution of host-language features as system innovations.

1. Critical Flaw in the Core "Cursor" Mechanism: The entire premise of reusable, stable scheduling libraries hinges on the stability of Cursors across transformations. However, the authors concede that this stability relies on "heuristic forwarding rules" (Section 5.2, page 7). This is a fatal flaw. The term "heuristic" implies that the behavior is not formally defined, is likely incomplete, and may be unpredictable. What are these heuristics? How are ambiguous cases—where a cursor points to code that is deleted, merged, or fundamentally altered—resolved? The paper provides no specification. Without a formal, predictable model for reference forwarding, the system's "safety" guarantees are moot. A user may receive a forwarded cursor that points to a semantically unrelated part of the program, leading to baffling errors. This is not a solid foundation for building robust libraries.

2. Grossly Unsubstantiated Claims of Code Reduction: The abstract claims an "order of magnitude" reduction in scheduling code. The evidence provided is wholly inadequate. In Figure 9a, the authors compare the lines of their Python scheduling code to the lines of generated C code. This is a nonsensical comparison. The correct baseline would be the lines of human-written source code in the reference libraries (e.g., the C and assembly code for the corresponding kernels in BLIS or OpenBLAS). Without this direct, apples-to-apples comparison, the claim of code reduction is unsubstantiated and appears to be an artifact of misleading metrics. The comparison in Figure 6c is similarly unconvincing as it compares a Python-based DSL to a C library without context.

3. Overstated Novelty by Conflating System and Host-Language Features: The authors repeatedly present standard metaprogramming features of Python as novel aspects of Exo 2. For instance, defining a reusable tile2D function (Section 3.2) or creating higher-order combinators like seq and repeat (Section 3.4) is trivial in any Python-hosted DSL. These are not contributions of the Exo 2 scheduling model; they are benefits of embedding a DSL in a powerful host language. This misattribution inflates the perceived novelty of the work and distracts from the actual (and, as argued above, flawed) core contribution of Cursors.

4. The "Competitive Performance" Claim is Not Rigorously Supported: While the system can generate fast code, the term "competitive" is used to paper over significant performance gaps. The heatmaps in Figure 8 clearly show many cases where Exo 2 is substantially slower than Intel MKL (e.g., dgemv_n, sgemv_t, dger), with performance penalties exceeding 30-50%. A rigorous evaluation would honestly report the geometric mean of these results and acknowledge where the system falls short, rather than making a blanket claim of competitiveness.

5. Reliance on Exception Handling for Control Flow: The promotion of try/except blocks as a primary mechanism for scheduling control flow (Section 3.3) is problematic. It encourages a "just try it and see" style of programming that obscures the actual preconditions required for a transformation to be valid. A more robust design would provide inspection primitives to query whether a transform can be safely applied, rather than forcing the user to catch a SchedulingError. This feels less like a feature and more like a workaround for insufficient static-checking capabilities.

Questions to Address In Rebuttal

The authors must address the following critical points for this work to be considered for publication:

1. On Cursor Forwarding: Please provide a formal specification or, at a minimum, a comprehensive and unambiguous description of the "heuristic forwarding rules" for Cursors mentioned in Section 5.2. How do you handle cases where a cursor's target is deleted or split? What guarantees, if any, can you provide about the semantic location of a forwarded cursor? Without this, the central mechanism of the paper lacks the necessary rigor.

2. On Code Reduction Claims: Please provide a rigorous, direct comparison to substantiate the "order of magnitude" code reduction claim. This requires measuring the lines of your Python scheduling code against the lines of human-written, high-performance C/assembly source code for the exact same set of kernel variants in a library like BLIS or OpenBLAS.

3. On Distinguishing Contributions: Please clarify which parts of the scheduling library design are enabled specifically by novel features in Exo 2, versus those that are a general property of using Python as a metaprogramming host language.

4. On Performance Evaluation: Please provide a more nuanced summary of the performance results from Figure 8, including geometric means across all problem sizes for each kernel. Can you explain the architectural reasons for the significant performance gaps versus MKL in cases like dgemv_n and dger?

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper presents Exo 2, a user-schedulable language (USL) designed around the central philosophy that scheduling languages should be extensible, or "growable," rather than providing a fixed set of compiler-defined scheduling operations. The authors argue that the diversity of hardware targets and optimization strategies makes it impossible for any fixed set of scheduling primitives to be universally sufficient.

The core contribution is a paradigm shift for USLs, moving from providing high-level scheduling directives to providing a set of fine-grained, trusted, equivalence-preserving primitives. Users can then compose these primitives in user-space code (Python) to build their own higher-level scheduling operations and libraries. The paper introduces a compelling conceptual framework for this, centered on three pillars: actions (code modification), inspection (code interrogation), and references (pointing to code). The technical enabler for this is a novel mechanism called Cursors, which provides stable, relative, and multiple references to object code that persist across transformations.

The authors demonstrate the power of this approach by building extensive scheduling libraries for high-performance computing, including a BLAS library covering over 80 kernels and a library that reproduces the complex scheduling behavior of Halide. Their results show that this approach can reduce scheduling code by an order of magnitude while achieving performance competitive with state-of-the-art, hand-optimized libraries like MKL and OpenBLAS.

Strengths

The primary strength of this paper is its powerful and well-articulated core thesis. It reframes the problem of USL design in a way that resonates deeply with historical lessons from programming language design and formal methods.

1. A Foundational Design Philosophy: The paper's argument for a "growable" language is not merely an engineering choice; it is a profound design philosophy. The authors connect their work to two foundational ideas in computer science: Guy Steele's "Growing a Language" and the LCF approach to theorem proving (Section 7, page 15). This is an exceptionally strong framing. Just as LCF provides a small set of trusted inference rules (axioms) from which complex proofs can be safely constructed, Exo 2 provides a small set of trusted scheduling primitives from which complex, semantics-preserving scheduling strategies can be built. This elevates the work from a description of a system to a proposal for a new, principled way to construct such systems.

2. A Clean Conceptual Model: The decomposition of scheduling language requirements into actions, inspection, and references provides a clear and valuable taxonomy. In particular, the detailed analysis of references (Section 5, page 5) is a significant contribution in its own right. The discussion of nominal vs. relative, single vs. multiple, and stable vs. one-time references provides the community with a much-needed vocabulary for comparing and contrasting the design of different meta-programming systems, from Halide to MLIR's Transform dialect.

3. Powerful and Novel Abstractions (Cursors): The Cursor mechanism is the key technical idea that makes the philosophy practical. The problem of maintaining valid references to a syntax tree while it is being actively transformed is a classic, difficult problem in meta-programming. The Cursor's support for stable, relative references and its well-defined forwarding semantics (Section 5.2, page 6) appears to be an elegant and powerful solution. This is what separates Exo 2 from prior systems that rely on brittle pattern matching or one-time handles that are invalidated after each transformation.

4. Compelling Empirical Evidence: The evaluation is thorough and convincing. It's one thing to propose a new language design; it's another to show that it can be used to build artifacts of significant complexity and performance. The creation of a BLAS library that rivals MKL and a library that can emulate Halide's sophisticated compute_at logic (Section 6.3.2, page 12) demonstrates that the primitive set is expressive enough for real-world tasks. The order-of-magnitude reduction in scheduling code (Figure 9, page 11) is a powerful testament to the abstraction power of the approach.

Weaknesses

The weaknesses are less about flaws in the work presented and more about the inherent trade-offs of the proposed approach and questions left for future work.

1. The Oracle Problem of Primitives: The entire LCF-style philosophy hinges on having a "correct" and "complete" set of axioms. The paper demonstrates that its chosen set of 46 primitives is sufficient for the case studies, but it doesn't offer a principled way to determine this set's sufficiency for future hardware or optimization patterns. What happens when a new accelerator requires a transformation that cannot be composed from the existing primitives? The user is then forced to modify the compiler, which is precisely the problem the system aims to avoid. The work moves the design burden from "what are the right high-level operations?" to "what are the right low-level primitives?", but the fundamental challenge of foresight remains.

2. The Meta-Programming Usability Gap: Debugging transformative meta-programs is notoriously difficult. When a user-defined scheduling function—a composition of dozens of fine-grained primitive calls—produces incorrect or suboptimal code, the debugging story is unclear. Tracing the state of the object program and the forwarding of multiple Cursors through a complex Python meta-program could be a significant challenge for users, potentially creating a new kind of productivity bottleneck.

3. Compositionality of Libraries: The paper masterfully demonstrates how to build powerful, domain-specific libraries (e.g., for BLAS). However, the vision of "growing a language" also implies the ability to compose abstractions from different libraries. What happens when a user wants to apply a scheduling function from a "tensor-core library" and another from a "memory-hierarchy library"? The paper does not explore the potential for interference or unexpected interactions between scheduling libraries developed by different parties.

Questions to Address In Rebuttal

1. Could the authors elaborate on the process used to arrive at the current set of 46 primitives? Was this set discovered reactively while building the case studies, or is there a more formal basis for its selection? What gives the authors confidence that this set provides a stable foundation for a wide range of future scheduling challenges?

2. Could the authors comment on the debugging experience for library authors in Exo 2? When a composite scheduling function like opt_skinny (Figure 7b, page 11) behaves unexpectedly, what tools or methodologies are available to inspect the intermediate states of the program and the validity of the Cursors?

3. The paper focuses on building self-contained scheduling libraries. What is the vision for how a typical user might compose or layer scheduling abstractions from different, independently-developed libraries? Does the Cursor mechanism and its forwarding semantics create any challenges for interoperability between libraries?

Review Form: The Innovator (Novelty Specialist)

---

Summary

The authors present Exo 2, a User-Schedulable Language (USL) designed around the principle of "growth." The central claim is that USLs should not be limited to a fixed set of scheduling operations provided by the compiler. Instead, they should provide a core set of trusted, fine-grained primitives that allow performance engineers to define their own, more powerful scheduling operations in user-level libraries.

To enable this, the authors identify three essential components for an extensible USL: actions (code modifications), inspection (code interrogation), and references (ways of pointing to code). They introduce a novel mechanism called Cursors to unify these concepts. Cursors are presented as a stable, multiple, and relative referencing system that, crucially, persists across code transformations via a "forwarding" mechanism. The paper demonstrates the power of this approach by building a scheduling library for linear algebra that significantly reduces the amount of scheduling code required for dozens of high-performance kernels.

---

Strengths

The primary strength of this paper lies in its novel and well-executed vision for how USLs should evolve.

1. A Novel Philosophy for USLs: The core idea of designing a USL to be "grown" by its users is a significant departure from the status quo. Prior USLs like Halide, TVM, and TACO focus on providing a powerful but ultimately fixed set of scheduling directives. Exo 2's philosophy of providing minimal, safe primitives to let users build their own directives is a novel and compelling paradigm shift. It directly addresses the scaling problem that arises when trying to schedule large libraries of related kernels.

2. The Cursor Mechanism as the Key Technical Novelty: The concept of a stable reference is the linchpin of the paper, and the Cursor mechanism is a genuinely new contribution in the context of USLs. It successfully synthesizes and improves upon ideas from prior art:

   • It allows for multiple references, like Halide's nominal system.
   • It allows for relative navigation (.parent(), .next(), etc.), drawing a clever analogy to DOM manipulation in systems like jQuery.
   • Most importantly, it provides temporal stability across transformations via the forwarding mechanism (Section 5.2). This is the critical delta compared to the one-time, invalidating handles of the MLIR Transform dialect or the single, one-time references in rewrite systems like ELEVATE. This stability is precisely what makes building reusable, composable scheduling libraries feasible.

3. Elevating Inspection to a First-Class Concept: While other systems may permit limited queries of the program structure, Exo 2 is the first USL I am aware of that explicitly frames inspection as a co-equal pillar alongside action and reference. By providing primitives to interrogate the object code (e.g., to implement bounds inference as a library function, as described in Section 4), the authors enable a more declarative and intelligent style of meta-programming, where scheduling libraries can make decisions based on the properties of the code they are transforming. This is a subtle but profound advancement.

---

Weaknesses

My critique is not that the work lacks novelty, but that its novelty could be framed more sharply against the most relevant prior art.

1. Novelty is in Synthesis, not de novo Invention: The individual ideas that compose Exo 2 have conceptual parallels elsewhere. Transformative meta-programming is the foundation of Lisp/Racket macros; tree traversal APIs are central to tools like jQuery and XPath/XSLT; and term rewriting with strategies is the core of languages like Stratego. The paper's novelty lies in the synthesis of these ideas and their specific, careful application to the domain of safe, performance-oriented USLs. The authors should be more explicit that the contribution is this novel synthesis, particularly the design of a reference system that satisfies the unique constraints of this domain.

2. Insufficient Differentiation from MLIR Transform Dialect in Core Sections: The MLIR Transform Dialect is arguably the closest prior art in a production compiler framework. While it is correctly identified and distinguished in the Related Work section (Section 7), the core technical sections (particularly Section 5 on Reference) would be much stronger if they used the MLIR handle mechanism as a direct foil to motivate the design of Cursors. This would make the novelty of the forwarding mechanism more immediately apparent to readers familiar with MLIR.

---

Questions to Address In Rebuttal

1. Robustness of Forwarding: The Cursor forwarding mechanism is the core of the stability claim. The paper mentions that heuristics are used for ambiguous cases (Section 5.2). Could the authors elaborate on the classes of transformations where forwarding might produce unintuitive or ambiguous results? For a library author, how predictable is the location of a cursor after a complex, user-defined scheduling operation is applied?

2. Completeness of Primitives: The authors have implemented a set of 46 scheduling primitives. Is there a principled basis for this set? How does one reason about its "completeness" for a given domain? For example, if a user wants to implement a novel data layout transformation, what is the process for determining if the existing primitives are sufficient, or if a new primitive must be added to the trusted compiler core?

3. Comparison to Traversal Strategies: The higher-order functions presented in Section 3.4 (e.g., repeat, reduce) bear a resemblance to the traversal strategies found in term rewriting systems (e.g., topdown, try in Stratego or ELEVATE). Could the authors clarify the relationship? Is the primary innovation the Cursor mechanism that provides a stable object on which to apply these strategies, or are there fundamental differences in the strategy combinators themselves?

Paper Title: Fast On-device LLM Inference with NPUs Reviewer: The Guardian (Adversarial Skeptic)

---

Summary

The authors present llm.npu, a system designed to accelerate the prefill stage of on-device LLM inference by offloading computation to the mobile Neural Processing Unit (NPU). The work identifies the prefill phase as the primary bottleneck in common mobile LLM tasks. To address this, the authors propose a three-level optimization strategy: (1) a "chunk-sharing graph" to handle variable-length prompts by splitting them into fixed-size chunks and sharing static operators; (2) a "shadow outlier execution" technique that partitions INT8-quantized matrix multiplications between the NPU and CPU to maintain accuracy by handling activation outliers on the CPU; and (3) an "out-of-order subgraph execution" scheduler to minimize pipeline bubbles between the heterogeneous processors. The authors claim significant improvements in prefill speed and energy efficiency over existing CPU and GPU-based systems.

Strengths

1. Problem Identification: The paper correctly identifies a critical and often-overlooked bottleneck in on-device LLM applications: the latency of the initial prompt processing (prefill) stage, especially for tasks requiring long contexts (Section 2.1, page 2). This is a well-motivated and timely problem.

2. Hardware Targeting: The core premise of leveraging the mobile NPU, a specialized but often underutilized processor for general LLM tasks, is sound. The micro-benchmarks in Section 2.2 (Table 3, page 4) effectively demonstrate the NPU's potential for INT8 matrix multiplication, establishing a solid foundation for the work's direction.

3. System-Level Approach: The authors have clearly undertaken a significant engineering effort to build a complete system. The work is not a single algorithmic trick but a combination of techniques designed to work in concert, which is commendable.

Weaknesses

My primary concerns with this manuscript center on the methodological soundness of key components, the rigor of the experimental validation against a critical baseline, and several seemingly contradictory or overstated claims.

1. Contradictory Claims Regarding Shadow Outlier Execution Overhead: The central premise of the "shadow outlier execution" is that it compensates for quantization errors with "minimal overhead" (Abstract, page 1). However, the authors' own analysis in Section 3.3 (page 7) directly contradicts this. They state: "...the synchronization of the reduced sum between CPU and NPU still takes non-trivial overhead, e.g., 29.7% end-to-end latency and 20.1% energy consumption on Qwen1.5-1.8B." This is not "minimal"; it is a substantial performance penalty. The proposed solution—pruning outliers from the "top 85% most unimportant layers"—is a heuristic that lacks sufficient justification. The choice of 85% appears arbitrary and its robustness across different models, tasks, and input distributions is not demonstrated. This core technique appears fundamentally flawed or, at best, its benefits are significantly overstated.

2. Insufficiently Rigorous Evaluation Against State-of-the-Art: The comparison against PowerInfer-v2 [94], the most relevant prior work that also utilizes mobile NPUs, is scientifically unsound. The authors explicitly state, "Since PowerInfer-v2 is not open-sourced, we use the reported data from its paper" (Section 4.1, page 9). Performance of such systems is intensely dependent on the specific hardware, OS version, and driver stack. Comparing llm.npu's performance on a Redmi K70 Pro against numbers reported in another paper for an unspecified or different device invalidates any claims of superiority. Without a direct, head-to-head comparison on the same hardware platform, the claimed 3.28-5.6x speedup is unsubstantiated.

3. Overstated Claim of Novelty: The abstract boldly claims llm.npu is "the first LLM inference system utilizing on-device Neural Processing Unit (NPU) offloading". Yet, the paper repeatedly cites PowerInfer-v2 [94] as a baseline that "also utilizes mobile NPUs to accelerate prefilling" (Section 4.1, page 9). This is a direct contradiction. The authors must be more precise in their contribution. Are they the first published system with a specific technique? The first to be open-sourced? The current claim is factually inaccurate as written.

4. Heuristics Presented Without Ablation or Sensitivity Analysis: The out-of-order subgraph execution scheduler (Section 3.4, page 8) relies on a greedy heuristic that prioritizes subgraphs based on a calculated "contribution" metric C. While the intuition is plausible, the paper provides no analysis of this scheduler in isolation. How does this heuristic compare to other potential scheduling strategies? How sensitive is performance to the specific formulation of C? The final performance gains are an aggregate of all three proposed techniques, making it impossible to assess the true efficacy of this scheduling approach on its own. The ablation study in Figure 19 (page 13) adds techniques sequentially, which does not isolate the scheduler's contribution from the benefits of the preceding optimizations.

Questions to Address In Rebuttal

1. Please reconcile the claim of "minimal overhead" for shadow outlier execution with your own measurement of a 29.7% latency overhead reported in Section 3.3. Furthermore, please provide a rigorous justification for the 85% outlier pruning threshold. How was this value determined, and how does its efficacy vary across the different models and datasets tested?

2. Given that a direct comparison against reported numbers from another paper is not a valid scientific comparison, how can the claims of superiority over PowerInfer-v2 be substantiated? Please provide a rationale for why a direct implementation or simulation of the PowerInfer-v2 approach was not attempted on your test hardware for a fair comparison.

3. Please clarify the paper's primary contribution with respect to novelty. In what specific way is this the "first" system of its kind, given the existence of PowerInfer-v2 which also targets NPUs for LLM prefill?

4. Can you provide a more detailed analysis of the out-of-order scheduling heuristic? A comparison against alternative, simpler scheduling policies (e.g., a baseline FIFO scheduler with overlapping) would strengthen the claim that your proposed heuristic is effective.

Paper Title: Fast On-device LLM Inference with NPUs Reviewer Persona: The Synthesizer (Contextual Analyst)

---

Summary

This paper presents 11m.npu, a novel system designed to accelerate the prefill stage of Large Language Model (LLM) inference on mobile devices by leveraging Neural Processing Units (NPUs). The authors correctly identify that for many emerging on-device applications with long contexts (e.g., UI automation, document summarization), the initial prompt processing (prefill) is a significant and often dominant bottleneck, a fact that has been relatively overlooked in favor of optimizing the token generation (decoding) phase.

The core contribution is a full-system, hardware-aware approach that re-constructs the prompt, model, and execution flow across three levels to make LLM inference "NPU-friendly." This involves: (1) dividing prompts into fixed-size chunks to overcome the NPU's limitation with dynamic shapes; (2) a "shadow outlier execution" technique that processes problematic activation outliers on the CPU/GPU in parallel, enabling the use of efficient per-tensor quantization on the NPU without sacrificing accuracy; and (3) an out-of-order scheduling algorithm for Transformer blocks to hide the latency of CPU/GPU-bound operations. The results are highly compelling, demonstrating an order-of-magnitude speedup in prefill latency and setting a new performance milestone of over 1,000 tokens/second for billion-parameter models on consumer hardware.

Strengths

1. High Significance and Timeliness: The paper addresses a critical and timely problem. As major industry players like Apple and Google push for on-device AI, the user experience of these features will be paramount. The authors provide a convincing analysis (Section 2.1, page 3) that prefill latency is a major obstacle to this vision, making their work immediately relevant to both the academic and industrial communities. This is not an incremental improvement; it's tackling a core barrier to practical deployment.

2. Pioneering System-Level Contribution: The most significant strength of this work is its framing as a complete system. Rather than proposing a single new algorithm, the authors present a holistic co-design that considers the interplay between the LLM architecture, quantization methods, and the specific constraints of mobile NPU hardware. This approach of pioneering the use of the NPU for LLM prefill opens a new and important direction for research in on-device AI.

3. Elegant Solution to the Quantization Dilemma: The "shadow outlier execution" (Section 3.3, page 7) is a particularly insightful contribution. The field of LLM quantization has struggled with the trade-off between accuracy and hardware efficiency. Fine-grained, per-group quantization preserves accuracy but maps poorly to accelerators like NPUs, while simple per-tensor quantization is efficient but suffers from outliers. This paper's solution—isolating the sparse, high-magnitude outliers for CPU/GPU processing while leaving the bulk of computation on the NPU—is a pragmatic and highly effective compromise. It connects the dots between the quantization literature (e.g., LLM.int8()) and the practical realities of heterogeneous mobile hardware.

4. Strong Empirical Evidence and Positioning: The experimental evaluation is thorough and the results are impressive. By demonstrating substantial speedups over strong baselines like MLC-LLM and PowerInfer-v2, the authors convincingly establish the effectiveness of their system. The achievement of >1000 tokens/sec prefill speed is a significant milestone that makes a strong statement about the potential of specialized mobile hardware.

Weaknesses

While the core ideas are strong, the work could be better contextualized and its boundaries explored more deeply.

1. Hardware Generality: The system is implemented and evaluated on Qualcomm Hexagon NPUs. While a reasonable choice given their prevalence, the paper would be strengthened by a discussion of how the core principles would translate to other mobile NPUs (e.g., from MediaTek, Google, or Apple). Are the identified challenges (static shapes, poor FP performance) universal? Are the proposed solutions (chunking, outlier handling) fundamentally applicable elsewhere, or are they deeply tied to the QNN framework and Hexagon architecture? A more abstract framing of the principles would broaden the work's impact.

2. Decoupling of Prefill and Decode Optimization: The paper makes a strong case for focusing on prefill and largely treats the decoding phase as a separate problem handled by a CPU backend. This is a pragmatic choice, but it leaves an interesting question unexplored: how can the system's scheduling and hardware utilization be optimized across both phases? The end-to-end latency results (Table 5, page 11) show that for tasks with longer outputs, the slower CPU decoding starts to matter more. This work provides the foundation, but the true "synthesized" system would co-schedule both phases across the CPU, GPU, and NPU holistically.

3. Assumptions about System Contention: The experiments are conducted in a controlled environment. A key challenge in mobile systems is resource contention, where the OS, UI rendering, and other background tasks compete for compute, memory bandwidth, and power. The paper acknowledges this is not considered (Section 4, page 9), but a discussion of the potential impact would be valuable. How robust is the out-of-order scheduler to unexpected stalls or CPU/GPU unavailability? This is a crucial step in bridging the gap between a research prototype and a deployable system service.

Questions to Address In Rebuttal

1. Could the authors elaborate on the fundamental principles of their approach that are likely to be portable across different NPU architectures, versus the optimizations that are specific to the Qualcomm platform? This would help clarify the generality of the contribution.

2. The paper focuses on accelerating the prefill phase, delegating the decoding phase to a CPU backend. Given the impressive results, what are the authors' thoughts on a more integrated approach? Could the out-of-order scheduling framework be extended to the decoding phase (e.g., for speculative decoding drafts) to further reduce end-to-end latency by leveraging the GPU or even the NPU?

3. Regarding real-world deployment, how might the performance of 11m.npu be affected by system-level resource contention on a mobile device (e.g., from concurrent applications or OS services)? Does the scheduler have any mechanisms to adapt to a dynamically changing execution environment?

Reviewer: The Innovator (Novelty Specialist)

Summary

This paper presents 11m.npu, a system designed to accelerate the prefill stage of on-device Large Language Model (LLM) inference by offloading computation to a mobile Neural Processing Unit (NPU). The authors identify the prefill phase as the primary bottleneck for on-device applications with long contexts. The core of their contribution is a three-level approach to re-structuring the model and prompt to make them amenable to efficient NPU execution: (1) a "chunk-sharing graph" technique to handle variable-length prompts by breaking them into fixed-size chunks and sharing static subgraphs; (2) a "shadow outlier execution" method that splits computation heterogeneously, running quantized operations on the NPU and sparse, high-precision outlier calculations on the CPU/GPU; and (3) an "out-of-order subgraph execution" scheduler to hide the latency of the CPU/GPU-bound operations.

While the paper presents a system that achieves impressive performance gains, my analysis focuses exclusively on the novelty of its core ideas. The central claim to novelty rests not on the use of NPUs for LLM prefill itself—which has been explored in prior work—but rather on the specific system architecture and set of optimizations designed to overcome the impedance mismatch between LLM workloads and existing mobile NPU hardware.

Strengths

The primary novel contribution of this work is the concept of "shadow outlier execution" (§3.3, page 7), which performs a heterogeneous decomposition of the LLM quantization problem. While handling activation outliers with mixed-precision execution is a known technique (e.g., LLM.int8() [33]), the decision to partition the problem across different processing units—bulk INT8 MatMul on the NPU and sparse FP32 outlier MatMul on the CPU—is a genuinely new system-level design pattern in this context. It directly addresses the architectural limitations of NPUs (poor FP performance) and leverages the strengths of the CPU. The subsequent optimizations, such as profiling for "hot channels" to manage memory, are clever engineering refinements built upon this core novel idea.

The "chunk-sharing graph" (§3.2, page 6) is a noteworthy engineering contribution. While its constituent elements—input tiling/chunking, static graph pre-compilation, and subgraph sharing—are all well-established principles in the compiler and systems domains, their synthesis and specific application to the Transformer architecture on mobile NPUs appear to be novel. The insight to distinguish between operators that are static (e.g., FFN) versus dynamic (e.g., Attention) with respect to chunk position and build a memory-efficient execution plan around this distinction is non-trivial and effective.

Weaknesses

The most significant weakness is the overstatement of novelty in the abstract. The paper claims to be "the first LLM inference system utilizing on-device Neural Processing Unit (NPU) offloading to reduce prefill latency." This claim is not accurate. The authors themselves cite PowerInfer-V2 [94] as the "most relevant work" (§6, page 14), which also utilizes mobile NPUs for prefill acceleration. Since the preprint for PowerInfer-V2 exists, 11m.npu cannot claim to be the "first." The novelty lies in the method, not the fundamental concept of using the NPU. The abstract and introduction should be revised to state the contribution with more precision, focusing on the novel architecture and techniques rather than a claim of primacy.

Furthermore, several of the core techniques are novel applications of existing concepts rather than fundamentally new ideas. The "out-of-order subgraph execution" (§3.4, page 8) is an application of classical task-graph scheduling on a Directed Acyclic Graph (DAG). The formulation of the problem is specific to this work, but the underlying principle is a cornerstone of computer science. The paper would be stronger if it explicitly framed this contribution as a novel application and heuristic for a known scheduling problem, rather than implying the invention of out-of-order execution in this context.

The complexity of the proposed system is substantial, involving multiple layers of offline profiling, graph partitioning, and a microsecond-level online scheduler. The ablation study (Figure 19, page 13) demonstrates that each component contributes to performance, but it is not entirely clear if the gains from the most complex components (e.g., the out-of-order scheduler) justify their complexity over simpler, more conventional heuristics. The delta between a sophisticated scheduler and a simpler greedy one that prioritizes available NPU tasks is not quantified.

Questions to Address In Rebuttal

1. Clarification on PowerInfer-V2: Please precisely articulate the novel contributions of 11m.npu over PowerInfer-V2 [94]. The related work section dismisses it by saying it doesn't "fully harness NPU capability," which is too vague. What specific, fundamental techniques presented in your paper are absent from or conceptually different than those in PowerInfer-V2?

2. Overhead of Heterogeneous Outlier Handling: The "shadow outlier execution" method is novel but introduces synchronization and data management overhead between the CPU and NPU. How does this overhead compare to the performance cost of handling outliers on a single, more flexible processor (like a GPU that can efficiently mix INT8 and FP16 operations)? Is there a point (e.g., a higher percentage of outliers) where this heterogeneous split becomes less efficient than a homogeneous approach?

3. Robustness of "Hot Channel" Profiling: The optimization of only caching weights for "hot channels" (§3.3, page 7) relies on offline profiling. How robust is this profile to shifts in domain or task? If the model is deployed for a new task where outliers frequently appear in previously "cold" channels, would the performance degrade significantly due to repeated disk/flash access for the corresponding weights?

4. Justification for Scheduler Complexity: The out-of-order scheduler employs a custom heuristic to minimize NPU stalls (§3.4, page 8). Could you provide evidence that this heuristic provides a significant performance advantage over a simpler baseline scheduler, such as a First-In-First-Out (FIFO) ready queue for each processor? This would help justify the added system complexity.

Paper Title: Faster Chaitin-like Register Allocation via Grammatical Decompositions of Control-Flow Graphs Reviewer Persona: The Guardian

---

Summary

The authors present a grammatical decomposition for control-flow graphs (CFGs) derived from structured programs. This decomposition, based on series, parallel, and loop operations, is claimed to precisely capture the structure of such CFGs, offering a more tailored alternative to general graph parameters like treewidth and pathwidth. Leveraging this decomposition, they develop dynamic programming algorithms for two variants of Chaitin-like register allocation: minimum-cost and spill-free. The primary claims are significant asymptotic runtime improvements over state-of-the-art treewidth/pathwidth-based methods and a practical implementation that scales to a higher number of registers (20) on real-world benchmarks than was previously feasible (8).

Strengths

1. Conceptual Elegance: The fundamental premise—that general-purpose graph decompositions like treewidth are overly broad for the specific sparsity found in CFGs—is sound and well-motivated. Tailoring the decomposition to the generative process of structured programs is a logical and elegant approach.

2. Asymptotic Improvement: The theoretical analysis presented in Section 3 demonstrates a clear and substantial asymptotic improvement. Reducing the dependency on the number of variables from |V|¹⁶·ʳ to |V|⁵·ʳ (for the min-cost problem, as stated in the Abstract and Section 3.1) is a non-trivial theoretical contribution, assuming the analysis is correct.

3. Strong Empirical Results on Selected Benchmarks: The experimental results in Section 4 are, on the surface, compelling. The ability to handle instances requiring up to 20 registers, where previous exact methods failed beyond 8, represents a significant practical breakthrough for the class of programs tested. Figure 11 (page 12) provides a stark visualization of the performance gains.

Weaknesses

My primary concerns with this manuscript revolve around overstated claims of generality and the representativeness of the experimental validation, which may inflate the perceived impact of the work.

1. Overstated Generality and Applicability: The authors claim their grammar "precisely captures the set of graphs that can be realized as CFGs of programs" (Abstract, page 1). This claim is demonstrably false in a general context. The grammar in Equation (1) (page 4) only models well-structured, goto-free programs. It completely omits common, real-world control-flow constructs like multi-level break/continue, general goto statements, and exception handling, all of which are present in languages like C, C++, and Java. This is a critical limitation that is only briefly acknowledged as a "major limitation" in the conclusion (Section 6, page 13). Such a fundamental constraint on the model should be stated clearly and upfront in the introduction and abstract, not relegated to future work.

2. Methodological Dependence on Source-Level Structure: The paper reveals in Section 4 (page 10) and Figure 6 (page 6) that the "grammatical decomposition" is obtained by directly parsing the source program. This is a crucial detail that weakens the overall contribution. The method does not decompose an arbitrary CFG; it leverages the parse tree of a well-structured program. This sidesteps the much harder problem of recognizing such a decomposition in a CFG that has been altered by prior compiler optimizations (e.g., code motion, inlining, loop transformations), which can obscure or destroy the simple nested structure the grammar relies on. The discussion on "Mixing with Other Compiler Optimizations" (Section 3.3, page 10) is cursory and unconvincing. It provides no evidence that the method remains applicable after aggressive, structure-altering optimizations that are common in production compilers.

3. Lack of Benchmark Diversity: The experimental validation relies exclusively on the SDCC regression test suite (Section 4, page 10), which consists of programs for embedded systems. Such programs are often small and highly structured, representing a best-case scenario for a grammar-based approach. The impressive performance results may not generalize to larger, more complex software systems (e.g., operating system kernels, large C++ applications, database engines) where control flow can be more intricate even without goto statements. The claims of solving "real-world instances" are therefore only substantiated for a very specific and favorable domain.

4. Potential for Prohibitive Constant Factors: The runtime complexity of O(|G|·|V|⁵·ʳ) still contains a term exponential in the number of variables, |V|. While r is treated as a constant in the analysis in Theorem 3.1, |V| is not. For functions with a large number of live variables, this term could dominate and render the algorithm impractical, even if the number of registers r is modest. The paper does not analyze or discuss this potential bottleneck, nor does it provide data on the maximum number of variables (|V|) in the benchmark functions.

Questions to Address In Rebuttal

1. Please clarify the paper's central claim of "precisely capturing" CFGs. Do you agree that this claim should be qualified to "CFGs of well-structured, goto-free, exception-free programs"? Please justify why this significant limitation is not emphasized earlier in the manuscript.

2. Your method relies on parsing the source code to obtain the decomposition. How would your approach handle a CFG whose structure has been significantly altered by a front-end or middle-end optimization pass, such as function inlining or loop unrolling, which may break the clean 1-to-1 mapping from your grammar? A concrete example would be instructive.

3. The performance claims are validated on a single benchmark suite for embedded systems. To substantiate the claim that this is the "first-ever exact algorithm that scales up to solve the real-world instances," please provide evidence of its performance on a more diverse set of benchmarks, particularly those known for larger functions and more complex (yet still structured) control flow, such as SPEC CPU or other general-purpose application suites.

4. The complexity O(|G|·|V|⁵·ʳ) is asymptotically better than previous work but retains a potentially large dependency on |V|. In your experiments, what was the distribution of |V| (the number of variables per function)? Did you encounter instances where the |V|⁵·ʳ term, rather than the r⁵ʳ term, became the practical performance bottleneck?

Paper Title: Faster Chaitin-like Register Allocation via Grammatical Decompositions of Control-Flow Graphs Reviewer Persona: The Synthesizer (Contextual Analyst)

---

Summary

This paper presents a novel approach to solving graph problems on Control-Flow Graphs (CFGs) by introducing a "grammatical decomposition" that precisely captures the structure of programs built from standard sequential, conditional, and loop constructs. The authors argue, correctly, that prior structural parameters like treewidth and pathwidth are overly general, as they characterize a much larger class of graphs than just CFGs. Their new decomposition serves as a more tailored and powerful tool for dynamic programming.

As a compelling application, the authors tackle the classical NP-hard problem of Chaitin-like register allocation. They develop a dynamic programming algorithm over their grammatical decomposition that is asymptotically and practically superior to state-of-the-art methods based on treewidth and pathwidth. Most notably, their experimental results demonstrate that their exact algorithm is the first to scale to the number of registers available in ubiquitous modern architectures (e.g., x86), a significant practical breakthrough.

Strengths

1. A Fundamental and Elegant Core Contribution: The central idea of this paper is not merely a faster algorithm but the foundational concept that enables it: a grammar that generates precisely the set of CFGs for structured programs. This moves beyond approximating CFG structure with generic graph parameters (treewidth) and instead provides a true, constructive model. This is a significant conceptual leap. The authors correctly connect this to the lineage of work on series-parallel graphs and graph grammars, but their extension to handle four terminals (Start, Terminate, Break, Continue) is a crucial and well-executed innovation that makes the theory applicable to real-world control flow.

2. Landmark Practical Implications for Register Allocation: While the theory is elegant, its application to register allocation is a "killer app." For decades, the community has treated Chaitin-style allocation as a problem requiring heuristics for any non-trivial instance. By providing an exact algorithm that scales to 20 registers, this work fundamentally challenges that long-held belief. The ability to find an optimal spill-free allocation for architectures like x86 (with 16 registers) is a remarkable achievement that could have a genuine impact on optimizing compilers, especially for embedded or performance-critical systems where spilling is highly detrimental.

3. High Potential for Broad Generality: The true strength of this work, from a synthesizer's perspective, is that register allocation is likely just the tip of the iceberg. The grammatical decomposition is a general-purpose tool. Any graph problem on CFGs that has previously been solved using dynamic programming over a tree or path decomposition is a candidate for re-evaluation with this superior structural foundation. This includes a wide array of problems in compiler optimization, program analysis, and model checking that the authors allude to in their introduction (Section 1, page 2). This paper doesn't just solve one problem; it provides a new lens through which to view a whole class of problems.

4. Excellent Contextualization and Clarity: The authors do a commendable job of positioning their work relative to the parameterized complexity literature, particularly the seminal work of Thorup [59] on the bounded treewidth of CFGs and more recent works [7, 25]. The paper is well-written, and the technical details of the decomposition are explained clearly with helpful figures (Figures 3-6, pages 5-6).

Weaknesses

1. Inherent Limitation to Structured Programs: The primary weakness of the approach is its strong reliance on the program being "structured" (i.e., reducible and largely goto-free). The grammar in equation (1) (page 4) does not account for arbitrary goto statements, exception handling (which introduces complex control flow), or other non-structured constructs found in languages like C, C++, or Java. While the authors acknowledge this limitation in their conclusion (Section 6, page 13), its significance is substantial. It confines the direct applicability of this powerful method to a (albeit large) subset of real-world code.

2. Understated Positioning Against Modern Compiler Design (SSA): The paper focuses on the classical "Chaitin-style" formulation of register allocation. Modern compilers (like LLVM and GCC) have largely moved to performing allocation on an SSA (Static Single Assignment) intermediate representation, where the problem is solvable in polynomial time (as noted in reference [9]). The authors correctly state that these are different problems, but they could strengthen their paper by better articulating the modern-day relevance and application domains of Chaitin-style allocation. For instance, is it for compilers that do not use SSA, for post-SSA stages, or for specific language targets? A more explicit discussion would help a broader compiler audience appreciate the impact of this work.

Questions to Address In Rebuttal

1. Regarding the limitation to structured programs: How brittle is the grammatical decomposition to minor deviations from pure structured control flow? Could the framework be extended to handle, for example, "mostly structured" programs with a few well-behaved gotos, perhaps by identifying and isolating the unstructured components?

2. Could you elaborate further on the target environments where an optimal, Chaitin-style register allocator would provide the most significant benefit today, considering the prevalence of SSA-based allocation? This would help frame the practical importance of your breakthrough results.

3. The loop composition operation (Figure 5, page 6) introduces new "boundary" vertices S, T, B, and C for the loop itself. Could you clarify how the live variable sets on these new vertices and their connecting edges are determined? Is the liveness analysis performed once on the final, composed CFG, or does it interact with the decomposition process itself?

Overall, this is an excellent paper with a powerful central idea and impressive results. The potential impact extends far beyond the specific application presented. I strongly recommend acceptance.

Reviewer: The Innovator (Novelty Specialist)

Review Form

Summary

This paper presents a grammatical decomposition for Control-Flow Graphs (CFGs) of structured programs. The authors argue that existing graph parameters like treewidth and pathwidth, while useful, are overly general for the specific structure of CFGs. They propose a new graph grammar, which they call SPL graphs, that precisely generates the set of CFGs for structured programs, including those with break and continue statements. The core of this grammar is a set of three composition rules (series, parallel, loop) operating on graphs with four distinguished vertices: Start, Terminate, Break, and Continue.

The authors then demonstrate the utility of this novel decomposition by applying it to two variants of Chaitin-like register allocation. They design dynamic programming algorithms based on their decomposition that achieve significant asymptotic runtime improvements over the state-of-the-art treewidth and pathwidth-based methods. The experimental results for spill-free register allocation are particularly compelling, showing that their method scales to 20 registers, whereas prior exact methods were limited to 8.

Strengths

The primary strength of this paper lies in its core conceptual contribution: the formulation of a graph grammar that is isomorphic to the syntactic structure of the programs it represents. While the general idea of using graph grammars or decompositions for program analysis is not new, the specific formulation presented here has a distinct and valuable novelty.

1. Precision of the Model: The central novel claim is the design of a decomposition that precisely captures the universe of structured CFGs. Prior art, such as treewidth/pathwidth decompositions, models a superset of these graphs. This paper's SPL grammar tightens the model to the exact domain of interest. This is a significant conceptual refinement.

2. The Four-Terminal Insight: The key mechanism enabling this precision is the use of four distinguished vertices (S, T, B, C). Standard series-parallel decompositions use two terminals (source, sink). The addition of dedicated terminals for break and continue flows is a simple but powerful idea that allows the grammar to elegantly and correctly model loop-related control flow, which has been a persistent challenge for simpler formalisms. This appears to be a genuinely new formulation in this context.

3. Significant Delta from Prior Art: The authors correctly position their work relative to prior art.

   • Vs. Treewidth/Pathwidth [7, 25, 59]: The "delta" here is specialization. Instead of a general-purpose decomposition that yields variable-sized cuts (bags), this work provides a domain-specific decomposition with a fixed-size cut (the four terminals). This is a textbook example of leveraging domain knowledge to create a more efficient structure, and the resulting asymptotic improvement from O(|V|¹⁶·ʳ) to O(|V|⁵·ʳ) is a direct and substantial consequence of this novel approach.
   • Vs. Series-Parallel Graphs [57, 60]: This work is a clear and non-trivial extension. It takes the foundational series and parallel operations and adds the crucial loop operation and the extra terminals needed to model programs.
   • Vs. Prior Graph Grammars [41, 61]: While the paper acknowledges these foundational works, the SPL grammar formulation appears more directly tied to modern programming language syntax (e.g., explicit if-then-else, while, break, continue). The clean homomorphism between the program's parse tree and the grammatical decomposition (as shown in Figure 6, page 6) is a strong indicator of a well-formed, novel abstraction.

Weaknesses

From a novelty perspective, the primary weakness is that the paper is evolutionary, not revolutionary. It builds upon a long history of using structural properties of graphs for algorithmic gain.

1. Incremental Nature: The idea of using grammars to define graph families is a classic concept. The contribution here is not the invention of graph grammars, but the creation of a particular grammar for a particular purpose. While I believe this specific grammar is novel and its application is impactful, one could argue it is an incremental step in a well-established research direction.

2. Limited Scope of Novelty: The novelty is strictly confined to structured, goto-free programs. The authors themselves acknowledge this limitation in Section 6. The proposed grammatical structure would likely collapse in the presence of arbitrary control flow (e.g., goto statements), as the fixed-terminal structure cannot model it. This boundary should be clearly understood as a delimitation of the paper's novel contribution.

3. Differentiation from Historical Context: The paper cites Kennedy and Zucconi [41], which also applied graph grammars to control flow analysis. While the authors' approach feels more modern and direct, the paper could benefit from a more explicit paragraph distinguishing the technical novelties of the SPL grammar from this specific piece of prior art. The current text on page 3 mentions it is "more general" in its coverage, but a more detailed technical comparison would strengthen the novelty claim.

Questions to Address In Rebuttal

1. Could the authors please provide a more direct and technical comparison to the graph grammar formalism in Kennedy and Zucconi [41]? What specific CFG structures, if any, can be generated by the SPL grammar that cannot be (or cannot be easily) generated by the grammar in [41], and vice-versa? Clarifying this would help solidify the "delta" over this important piece of prior art.

2. The four terminals (S, T, B, C) are the cornerstone of the novel decomposition. Is there a formal argument for why this set is both sufficient and minimal for representing the CFGs of the program class defined in grammar (1) on page 4? For instance, why isn't a single "exit" terminal sufficient for both break and continue? (The answer seems obvious from the graph structures, but a formal justification would be valuable).

3. The proposed decomposition seems inherently tied to the syntax of the source program. If a program is destructured by a series of compiler optimizations (before register allocation), can a grammatical decomposition still be recovered from the resulting CFG? Or is the novelty of this approach dependent on performing it early in the compilation pipeline, using the program's parse tree?

Paper Title: FleetIO: Managing Multi-Tenant Cloud Storage with Multi-Agent Reinforcement Learning Reviewer: The Guardian

---

Summary

The paper presents FleetIO, a framework using multi-agent reinforcement learning (MARL) to manage virtualized SSDs in a multi-tenant environment. The stated goal is to resolve the long-standing tension between performance isolation and resource utilization. The authors propose a MARL formulation where each virtual SSD (vSSD) is controlled by an RL agent. The core contributions include: 1) a specific RL state, action, and reward formulation for vSSD management; 2) a "ghost superblock" (gSB) abstraction to facilitate fine-grained, dynamic resource harvesting between vSSDs; and 3) an evaluation on a programmable SSD demonstrating improved utilization and tail latency compared to several baseline approaches.

---

Strengths

1. Problem Significance: The paper addresses a fundamental and well-understood problem in cloud storage virtualization—the trade-off between strict hardware isolation and efficient software-based sharing. The motivation is clear and compelling.

2. Systems Implementation: The work is grounded in a real-system implementation on a programmable, open-channel SSD. This is a significant strength over purely simulation-based studies and adds considerable weight to the performance results.

3. Concrete Abstraction: The proposed "ghost superblock" (gSB) is a tangible systems-level contribution. It provides a mechanism to realize the policies decided by the RL agents, moving beyond a purely algorithmic proposal.

---

Weaknesses

My primary concerns with this submission relate to the rigor of the RL formulation, the limited scale of the evaluation, and the potential for unstated complexities and overheads.

1. Arbitrary Reward Formulation: The core of the RL system, the reward function (Section 3.3.3, Equations 1 & 2), appears to be a work of meticulous, yet arbitrary, manual engineering rather than emergent learning. The function is critically dependent on hyperparameters α and β. The paper states β is set to 0.6 "based on our study" (page 6), but this study is not presented. Similarly, the per-cluster α values are derived from a search. This raises a significant question: has the system truly learned an optimal policy, or has it simply been hand-tuned through these coefficients to achieve the desired outcome on a specific set of workloads? The very premise of using RL is to avoid such manual tuning, yet the system's success seems to hinge upon it.

2. Insufficient Scalability Validation: The scalability evaluation in Section 4.3 is unconvincing. The experiments are limited to a maximum of 8 vSSDs. A modern cloud SSD could host dozens of low-intensity tenants. The paper claims FleetIO "consistently improves" utilization as vSSDs increase, but the data in Figure 14(a) shows the improvement factor over Hardware Isolation decreases from 1.33x (4 vSSDs) to 1.18x (8 vSSDs). This suggests that as contention and complexity increase, the benefits may diminish. The MARL coordination, which relies on shared state, could easily become a bottleneck at a realistic scale. The claim of scalability is not sufficiently supported.

3. Unaccounted Overheads of Harvesting: The gSB abstraction, while clever, introduces significant complexity, particularly its interaction with garbage collection (Section 3.7). The process of migrating valid data from harvested blocks back to a vSSD's primary blocks during GC (Figure 9) is a form of data movement that will inevitably incur write amplification and latency. The paper dismisses this with a claim of "< 5% write amplification" (page 8) without providing the methodology or data to substantiate it. Under what conditions was this measured? How does this behave under a write-heavy, GC-intensive workload mix? I suspect there are corner cases with severe performance degradation that have not been presented.

4. Fragility of Workload Clustering: The system's performance relies on pre-classifying workloads into types to apply fine-tuned reward functions (Section 3.4). This clustering is performed on a small, static set of 9 workloads. The real world is not so clean. The paper's solution for a novel workload is to fall back to a "unified reward function" and mark it for offline re-tuning. This implies a potentially significant period of suboptimal performance for any new application. The robustness claim in Section 4.6, which only swaps between known workload types, does not adequately address the "cold start" problem for a truly novel workload.

---

Questions to Address In Rebuttal

The authors must address the following points to make this work convincing:

1. On the Reward Function: Please provide a thorough sensitivity analysis for the α and β hyperparameters. How much does performance degrade if, for instance, β is set to 0.4 or 0.8? Justify the claim that the chosen values are optimal beyond the specific workloads tested. The current presentation makes the reward function seem more like a brittle, manually-tuned heuristic than a robust, learned policy.

2. On Scalability: The claim of scalability is a primary concern. Can you provide either experimental data (even from a scaled-up simulation, if hardware is limited) or a detailed theoretical argument for why the MARL coordination mechanism and gSB management will not become a performance bottleneck with 16, 32, or more tenants on a single device?

3. On Garbage Collection Overhead: The claim of negligible (< 5%) write amplification from gSB-related data migration during GC needs substantiation. Please provide detailed data showing the WA and P99.9 latency during GC cycles for a worst-case scenario (e.g., multiple write-intensive tenants actively using harvested blocks when the host vSSD's GC is triggered).

4. On Generalization to Novel Workloads: What is the measured performance difference between a workload running with its "optimized" reward function versus the "unified" reward function it would use upon first deployment? This is critical to understanding the practical cost of the system's reliance on pre-training and clustering.

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper presents FleetIO, a reinforcement learning (RL) based framework for managing multi-tenant virtualized SSDs in a cloud environment. The work directly addresses the long-standing and fundamental trade-off between performance isolation and resource utilization. Traditional hardware-isolated approaches provide strong performance guarantees but lead to underutilization, while software-isolated approaches improve utilization at the cost of performance interference and tail latency.

FleetIO's core contribution is the co-design of a multi-agent reinforcement learning (MARL) policy with a novel system abstraction, the "ghost superblock" (gSB). Each virtual SSD (vSSD) is managed by an independent RL agent that learns to dynamically "harvest" or "make harvestable" storage bandwidth from its peers. The gSB abstraction provides the necessary system-level mechanism to track and manage these fine-grained, harvestable resource blocks transparently. The system further enhances its effectiveness by clustering workloads and fine-tuning the RL reward functions for different workload types (e.g., latency-sensitive vs. bandwidth-intensive). The experimental evaluation, conducted on a real programmable SSD, demonstrates that FleetIO can significantly improve storage utilization (up to 1.4x) compared to hardware-isolated approaches while simultaneously reducing tail latency (by 1.5x) compared to software-isolated approaches, effectively achieving a superior point in the design space that was previously unattainable.

Strengths

1. Addresses a Fundamental and High-Impact Problem: The tension between isolation and utilization is not a niche issue; it is a central challenge in the design of cost-effective and performant multi-tenant cloud systems. By tackling this problem head-on, the paper's potential impact is substantial. A solution that can reclaim underutilized resources without sacrificing SLOs is of immense practical value to any cloud provider.

2. Elegant Co-Design of Learning and Systems: This paper is an excellent example of a true "ML for Systems" work. The authors did not simply apply an off-the-shelf RL algorithm to a system problem. Instead, they recognized that the learning agent needed a proper "actuator" to enact its decisions. The development of the ghost superblock (gSB) abstraction (Section 3.6, page 7) is a key insight. It provides a clean, manageable interface that decouples the high-level policy decision ("harvest 100 MB/s of bandwidth") from the messy low-level details of physical block management. This synergy between the learning framework and the system abstraction is the paper's greatest strength.

3. Pragmatic and Well-Justified RL Formulation: The choice of a multi-agent system with independent learners is well-suited for scalability. The reward function (Section 3.3.3, page 6) is thoughtfully constructed to balance bandwidth gains against SLO violations, directly encoding the paper's core objective. Furthermore, the decision to cluster workloads and fine-tune the reward function's trade-off parameter (α) is a pragmatic recognition that a single reward function is unlikely to be optimal for all application types. This demonstrates a mature understanding of both the system's needs and the practical application of RL.

4. Strong and Convincing Evaluation: The evaluation is thorough and well-designed. The authors compare FleetIO against a comprehensive set of baselines, including static hardware/software isolation, a more recent DNN-based approach (SSDKeeper), and a heuristic adaptive method. The results presented in Figure 10 (page 10) compellingly illustrate how FleetIO carves out a new, superior position in the utilization-vs-latency trade-off space. The scalability experiments (Section 4.3, page 10) and the reward function ablation study (Section 4.4, page 11) further strengthen the paper's claims.

Weaknesses

While the work is strong, there are opportunities to further contextualize its contributions and consider its broader implications. These are not flaws so much as avenues for deeper discussion.

1. The Brittleness of the Reward Function: The paper demonstrates that fine-tuning the reward function's alpha parameter is critical for performance (Figure 15, page 11). This highlights the power of the approach but also hints at a potential fragility. In a real-world cloud environment with an ever-changing mix of novel workloads, the process of defining clusters and manually tuning these hyperparameters could become a significant operational burden. The work could be strengthened by discussing the sensitivity to these parameters and exploring whether the agents could learn this trade-off themselves, perhaps through a meta-learning or hierarchical RL approach.

2. Scope of the RL State Space: The state representation defined in Table 1 (page 5) is reasonable and effective for the task at hand. However, it omits longer-term device health metrics, most notably flash endurance (wear). The agents' policies, by shifting I/O patterns to harvest bandwidth, will inevitably impact the write distribution across flash blocks. A policy that aggressively utilizes certain channels could lead to premature wear. The current framework is blind to this, which could be a significant concern in a production deployment spanning years. This work opens a fascinating future direction where the RL agent must also optimize for device lifetime.

3. Simplicity of Inter-Agent Coordination: The MARL coordination is achieved via a simple term in the reward function that considers the average reward of other agents (Equation 2, page 6). This is an effective and scalable approach. However, the systems community is increasingly exploring more complex coordination strategies. It would be valuable to discuss why this simple implicit coordination was chosen over more explicit methods (e.g., a centralized critic or direct agent-to-agent communication) and what the potential trade-offs might be.

Questions to Address In Rebuttal

1. Regarding the reward function tuning (Section 3.4, page 6): How sensitive is the overall system performance to the alpha and beta hyperparameters? How would a cloud provider be expected to set these values in a large-scale deployment where new, un-clustered workload types appear frequently?

2. The paper focuses on the immediate performance trade-offs of latency and bandwidth. Could the authors comment on how FleetIO's dynamic harvesting might affect long-term SSD health, specifically write amplification and wear-leveling? Is it possible for an agent to learn a "parasitic" policy that improves its own metrics by prematurely aging a peer's portion of the SSD?

3. The multi-agent coordination mechanism is simple and elegant. Could the authors briefly discuss if they considered more complex MARL coordination schemes and elaborate on their decision to use the current shared reward formulation? What benefits or drawbacks might a more complex approach introduce in this specific systems context?

Paper Title: FleetIO: Managing Multi-Tenant Cloud Storage with Multi-Agent Reinforcement Learning Reviewer Persona: The Innovator (Novelty Specialist)

---

Summary

This paper presents FleetIO, a framework that applies multi-agent reinforcement learning (MARL) to manage multi-tenant virtualized SSDs. The central goal is to break the long-standing tradeoff between performance isolation (favored by hardware-isolated approaches) and resource utilization (favored by software-isolated approaches). The authors propose a MARL formulation where each virtual SSD (vSSD) is controlled by an RL agent that can take actions like harvesting idle bandwidth from other vSSDs or adjusting its own I/O priority. To enable this, the paper introduces a new systems-level abstraction called the "ghost superblock" (gSB) to track and manage harvestable storage blocks. The authors also propose fine-tuning the RL reward functions based on workload types, which are identified at runtime using a clustering approach. The system is implemented and evaluated on a real programmable SSD, demonstrating significant improvements in both utilization and tail latency compared to state-of-the-art approaches.

Strengths

1. Novelty of Application and Synthesis: The primary strength of this work lies in its novel application of a known technique (MARL) to a persistent and important systems problem. While RL has been used for resource management in other domains (e.g., network scheduling, job scheduling), the authors' claim in Section 2.3 (page 4) to be "the first work to investigate RL in virtualized storage resource management" appears to be accurate. The synthesis of MARL with the specific challenges of SSD virtualization—including I/O interference, garbage collection, and dynamic workloads—represents a genuinely new approach in this space.

2. A Novel Systems Abstraction to Support the Learning Framework: The proposed "ghost superblock" (gSB) abstraction (Section 3.6, page 7) is a significant and novel systems-level contribution. Many "ML for Systems" papers apply a learning model as a black box without deeply considering its integration into the underlying system. Here, the authors have designed a new data structure and management layer specifically to translate the high-level decisions of the RL agents (e.g., "harvest X MB/s of bandwidth") into concrete, low-level actions on flash blocks. This tight co-design between the learning algorithm and the system architecture is a clear point of novelty.

3. Addresses a Fundamental, Non-Incremental Problem: The paper does not target a marginal improvement. It directly confronts the fundamental tension between isolation and utilization in shared storage, a problem that has existed for decades. By demonstrating a solution that can simultaneously improve utilization by up to 1.4x and decrease tail latency by 1.5x (as claimed in the abstract), the work presents a paradigm shift away from the static or purely heuristic-based methods of the past. The results shown in the tradeoff graph (Figure 10, page 10) compellingly illustrate that this new approach occupies a previously unattainable point in the design space.

Weaknesses

1. Limited Novelty in the RL Methodology Itself: While the application of RL is novel, the specific RL techniques employed are standard. The paper uses Proximal Policy Optimization (PPO), a well-established algorithm, and a multi-agent formulation based on independent learners that observe some shared state and use a linearly combined reward function (Equation 2, page 6). There is no new contribution to reinforcement learning theory or multi-agent coordination algorithms. The novelty is therefore confined to the application domain and systems integration, not the core learning method. This should be made clearer.

2. The "Workload Clustering" is Functionally Similar to Prior Work: The idea of classifying workloads to apply different policies is not new. While the use of unsupervised clustering (Section 3.4, page 6) is a reasonable approach, it is conceptually similar to prior systems that identify workload characteristics (e.g., latency-sensitive vs. bandwidth-intensive) to apply different QoS policies or scheduling rules. The novel element here is that the output of the clustering informs the selection of an RL reward function, but the act of classification itself is a well-trodden path.

3. Potential Overstatement of "Automated" Decision-Making: The system relies on several key hyperparameters that appear to be manually tuned, which tempers the claim of a fully automated solution. For instance, the reward balancing coefficient β is set to 0.6 "by default based on our study" (Section 3.3.3, page 6), and the SLO violation threshold is set to 5% during fine-tuning (Section 3.4, page 7). A truly novel, learning-based system would ideally learn these tradeoffs or be robust to their settings. The sensitivity to these choices is not explored, making it unclear how much expert tuning is required to achieve the reported results, a common issue when moving a complex learning system into practice.

Questions to Address In Rebuttal

1. On the Scope of Novelty: The paper's novelty rests on being the first to apply RL to this specific storage problem. Could the authors please situate their contribution more precisely with respect to the broader literature on using RL for resource management in other cloud infrastructure domains (e.g., memory management, network traffic engineering, or CPU scheduling)? Is the core challenge here fundamentally different, or is this primarily a successful porting of the RL-for-resource-management paradigm to a new domain? A clear characterization would strengthen the paper.

2. On the Necessity of the gSB Abstraction: The ghost superblock (gSB) is presented as a core contribution. Was this new abstraction strictly necessary to implement dynamic bandwidth harvesting? Could you discuss alternative, perhaps simpler, mechanisms you considered for tracking and lending flash blocks between vSSDs? For example, could this have been managed with simpler metadata tables without introducing a new "superblock" concept? Justifying this specific design choice over others would bolster its claim as a significant and necessary innovation.

3. On the Novelty of the Multi-Agent Formulation: The multi-agent reward function uses a fixed, manually-set parameter (β) to balance individual vs. system-wide goals. This is a common heuristic in multi-agent RL. Are there more advanced or adaptive coordination mechanisms from the MARL literature that could have been applied? A discussion of why this simpler, non-learning coordination mechanism was chosen would help clarify whether it is sufficient for this problem or simply a first step.

Title: Forecasting GPU Performance for Deep Learning Training and Inference Reviewer: The Guardian (Adversarial Skeptic)

---

Summary

The authors present NEUSIGHT, a framework for forecasting the performance of deep learning workloads on GPUs, with a particular focus on predicting latency for unseen models and hardware. The core methodology deviates from prior work by not predicting kernel latency directly. Instead, it decomposes kernels into smaller units called "tiles," uses a Multi-Layer Perceptron (MLP) to predict the hardware utilization for a single tile, constrains this prediction using fundamental performance laws (i.e., roofline), and then aggregates these per-tile estimates to derive the end-to-end kernel latency. The authors claim this approach significantly reduces prediction error compared to state-of-the-art methods, citing a dramatic improvement from 121.4% to 2.3% for GPT-3 training on an H100 GPU.

Strengths

1. Sound De-construction of the Problem: The high-level insight to decompose the complex problem of kernel latency prediction into smaller, more manageable units (tiles) is conceptually strong. Predicting a bounded utilization value is a more constrained problem for a machine learning model than predicting an unbounded latency value directly, which could contribute to better generalization.
2. Extensive Empirical Evaluation: The paper is evaluated on a comprehensive set of modern GPUs (including H100, L4, and AMD MI-series) and relevant large language models (BERT, GPT variants, etc.). The inclusion of out-of-distribution test cases for both hardware and models is a necessary and welcome component of the evaluation.
3. Clarity of Presentation: The paper is generally well-written, and the overall workflow of the NEUSIGHT framework is clearly articulated, particularly in Figure 6.

Weaknesses

My primary concerns with this paper stem from critical methodological assumptions that appear to be either unjustified or oversimplified, and a potential overstatement of the framework's capabilities, especially concerning "unseen" scenarios.

1. The "Oracle" of Tile Dimensions: The entire framework is predicated on knowing the tile dimensions for a given kernel on a target GPU. The authors state that "The tile dimensions are determined by metadata obtained with PyTorch Profiler" (Section 4.1, page 7) and for prediction, they "estimate tile sizes by finding the closest match in the database" (Section 6.1, page 10). This is a critical flaw that undermines the central claim of predicting performance on unseen GPUs and for new models.

   • This approach is not predictive; it is reactive. It requires a pre-existing, comprehensive database of profiled tile configurations. If a new version of cuDNN or CUTLASS introduces a novel tiling strategy, or if a truly new kernel is developed, this database would contain no relevant entry. The system would be unable to make a prediction. This is a significant limitation that is not adequately addressed. The framework does not predict performance from first principles of the hardware and kernel, but rather pattern-matches against previously observed implementations.

2. Unjustified Functional Form for Utilization: The core of the prediction model relies on Equations 7 and 8, which model utilization as utilization = alpha - beta / num_waves. The paper provides no theoretical or microarchitectural justification for this specific hyperbolic relationship. This functional form appears to be an arbitrary, empirical curve-fit to the data shown in Figure 5. While it may fit the observed data, there is no guarantee it will generalize to different kernel types, new GPU architectures with different scheduler designs, or memory subsystems. The claim of "imposing performance laws" is therefore weak; the framework imposes an assumed curve whose parameters are predicted by an MLP, and then multiplies the result by a performance bound.

3. Oversimplified Latency Aggregation: Equation 4 (PerOpLatency = PerTileLatency × num_waves) presents a grossly simplified model of execution. It assumes that waves of tiles execute in a perfectly sequential and independent manner. This model completely ignores:

   • Pipeline stalls and overheads between waves.
   • Memory contention that may not scale linearly with the number of waves.
   • Complex hardware scheduling effects where the GPU might overlap execution of tiles from different waves or manage resources in a non-linear fashion. This assumption is a significant abstraction that is unlikely to hold true in all cases, yet it is presented without validation or sensitivity analysis.

4. Inconsistent and Potentially Misleading Error Reporting: The abstract and conclusion highlight exceptionally low error rates (e.g., 2.3%). However, a closer inspection of the results reveals a more complex picture. Table 7 (page 12) on operator fusion shows prediction errors as high as 24.6% for BERT-Large and 19.4% for GPT2-Large on the H100. Operator fusion is a standard and critical optimization in modern deep learning frameworks. The fact that the model's error rate increases by an order of magnitude for these common cases suggests a fundamental weakness in the methodology when dealing with kernels that deviate from the simple, well-structured operations used to build the core models. These high-error results are not reconciled with the headline claims.

5. Trivial Distributed Performance Model: The extension to distributed execution (Section 5.1, page 9) is superficial. The network performance estimation is a basic analytical model based on peak bandwidth and a utilization factor. The claim that NEUSIGHT can be integrated with simulators like ASTRA-Sim is an assertion, not a contribution. Furthermore, the multi-node results presented in Table 9 are unvalidated estimations, which adds little value to the paper's core claims.

Questions to Address In Rebuttal

1. On Tile Dimensions: The methodology relies on a pre-populated tile database from a profiler. How does the framework handle a genuinely novel kernel from a new library (e.g., a future version of FlashAttention) whose tiling strategy is not represented in the training data? How can the claim of forecasting for "unseen models" be justified if the underlying kernel implementations must be known a priori?

2. On the Utilization Model: Please provide a theoretical or microarchitectural justification for the specific functional form utilization = alpha - beta / num_waves. Why was this form chosen over other potential models? Have the authors tested its robustness on kernels where performance does not scale smoothly with concurrency?

3. On Latency Aggregation: The sequential wave execution model (Equation 4) is a strong simplification. Can the authors provide evidence from hardware counters or microbenchmarks that this model holds true across different architectures and that it does not introduce significant, unaccounted-for error, especially for memory-intensive kernels?

4. On Inconsistent Error Rates: Please explain the significant discrepancy between the low overall errors reported in Figure 7 and the much higher errors for fused operators in Table 7 (e.g., ~24% for BERT/GPT-2 on H100). Does this suggest a fundamental limitation in modeling kernels that deviate from standard GEMM structures?

5. On Generalization to New Libraries: How would NEUSIGHT's performance be affected by a major update to a kernel library like cuDNN, which might fundamentally change the tiling strategies for a given operation and GPU? Would the entire tile database and MLP models need to be regenerated and retrained, thus limiting the framework's practical utility for forward-looking predictions?

Reviewer: The Synthesizer (Contextual Analyst)

Summary

The authors present NEUSIGHT, a framework for forecasting the performance of deep learning workloads on GPUs, with a particular focus on predicting latency for unseen models on unseen hardware. The core contribution is a methodological shift away from prior work that uses monolithic machine learning models to directly predict end-to-end kernel latency. Instead, NEUSIGHT leverages a key architectural insight: modern GPU libraries execute large kernels by decomposing them into smaller, independent work units called 'tiles'.

The framework's prediction process is decomposed accordingly. It first predicts performance at the tile granularity. Crucially, it does not predict latency directly. Instead, it uses a small MLP to predict the hardware utilization, a more constrained and physically grounded variable. This prediction is then used within the framework of fundamental performance laws (i.e., a Roofline model) to calculate tile latency. These tile-level predictions are then aggregated to estimate the latency of the full kernel and, subsequently, the entire model. The authors demonstrate through a comprehensive evaluation that this architecturally-aware, hybrid approach dramatically reduces prediction error compared to state-of-the-art baselines, especially in challenging out-of-distribution scenarios involving new GPUs like the H100.

Strengths

1. Principled Hybrid Approach: The paper's primary strength is its departure from treating performance prediction as a pure black-box regression problem. By grounding the prediction in the physical reality of GPU execution—tiled decomposition and performance bounds defined by FLOPs and memory bandwidth—the authors create a model that is far more robust and generalizable. Using machine learning for the component that is hardest to model analytically (the non-linear relationship between workload size and hardware utilization) while relying on established performance laws for the rest is an elegant and powerful synthesis.

2. Excellent Generalization: The most compelling evidence for the framework's success is its remarkable accuracy on out-of-distribution workloads and hardware. The results for predicting GPT-3 performance on the H100 GPU (Section 6.2, page 10), where neither was part of the training set, are particularly impressive. This demonstrates that the model has learned something fundamental about the relationship between kernel parameters and GPU architecture, rather than simply overfitting to a specific set of hardware. The extension to AMD GPUs (Figure 9, page 12) further strengthens this claim, suggesting the underlying principles are vendor-agnostic.

3. Significant Advancement over Prior Art: The paper does an excellent job of positioning its contribution. It not only cites prior work but empirically demonstrates its limitations (Figure 2, page 3). By showing that both linear regression and more complex MLP-based approaches fail to generalize, the authors build a strong case for why a new methodology is needed. The orders-of-magnitude reduction in prediction error presented in the evaluation is not an incremental improvement; it represents a significant step forward for the field.

4. High Potential Impact: This work addresses a critical and commercially relevant problem. The ability to accurately forecast the performance of new, large models on next-generation or access-constrained hardware is invaluable for hardware procurement, cloud resource allocation, and ML model co-design. NEUSIGHT provides a practical tool that could directly influence billion-dollar decisions for large technology companies and research institutions. The framework's ability to plug into larger system simulators for distributed training (Section 6.3, page 12) further broadens its utility.

Weaknesses

While the work is strong, its long-term viability hinges on a few assumptions that could be points of fragility.

1. Dependence on Tile Metadata: The entire methodology is predicated on the ability to extract or infer tile dimensions from kernel metadata (e.g., kernel names from the PyTorch Profiler, as mentioned in Section 6.1, page 10). This dependency is a potential weakness. Future compilers or GPU libraries could easily change naming conventions, obfuscate this information, or adopt more dynamic tiling strategies, which might break the current extraction process and require significant re-engineering.

2. Extensibility to Novel Operator Types: The framework uses specialized MLPs for different classes of common DNN operators. While the paper mentions a fallback strategy for unknown operators (treating them as memory-bound), this is a coarse approximation. The true test of a predictive model is its ability to handle novelty. The performance on fundamentally new types of kernels—for instance, those arising from sparse computation, graph-based models, or non-transformer architectures—remains an open question.

3. Simplified Aggregation Model: The model aggregates tile latencies by assuming sequential execution of "waves" of tiles (Equation 4, page 7). While this seems to work exceptionally well, it abstracts away complex microarchitectural interactions, such as L2 cache contention between concurrently executing tiles or memory controller scheduling effects. The current success might be partially attributable to the regular, dense nature of transformer workloads. The model might be less accurate for workloads that create more resource contention between tiles.

Questions to Address In Rebuttal

1. Could the authors comment on the robustness of their tile-size extraction methodology? How might the NEUSIGHT framework adapt if future GPU software stacks (e.g., CUDA 14, new versions of cuDNN) were to change kernel naming conventions or make this metadata less accessible?

2. The multi-node execution predictions for a large-scale GPT-3 deployment (Table 9, page 13) are a fascinating projection but could not be validated. Beyond the network model itself, what aspect of scaling from a single server to thousands of nodes do the authors believe introduces the most uncertainty into their predictions? For example, are there concerns about tail latency effects or interactions between compute and communication that are not captured in the current model?

3. While the framework handles various standard DNN kernels, could you elaborate on its limitations when faced with operators that have highly irregular memory access patterns or control flow? How would the concept of a uniform 'tile' and the associated Roofline bounds apply in such scenarios?

Paper Title: Forecasting GPU Performance for Deep Learning Training and Inference Review Form: The Innovator

---

Summary

The authors present NEUSIGHT, a framework designed to forecast the performance of deep learning models on unseen GPUs. The central claim to novelty lies in the framework's core methodology. Instead of applying machine learning directly to predict the end-to-end latency of a full DNN kernel—a common approach in prior work—NEUSIGHT decomposes the problem. It identifies that GPU libraries execute large kernels as a collection of smaller, independent work units, which the authors term 'tiles.' The framework's key contribution is to predict performance at this finer tile granularity. Specifically, it uses a Multi-Layer Perceptron (MLP) not to predict a raw latency value, but to predict a utilization factor. This predicted utilization is then used to scale a theoretical performance bound derived from fundamental hardware limits (i.e., the roofline model). These tile-level predictions are subsequently aggregated to produce a forecast for the entire kernel and, ultimately, the full model.

Strengths

The primary strength of this paper is the novelty of its core architectural insight. It moves beyond the prevalent black-box, end-to-end prediction paradigm for GPU kernels and introduces a more physically-grounded, gray-box approach. The specific points of novelty are:

1. Problem Decomposition: The decision to model performance at the tile level, rather than the kernel level, is a significant departure from cited prior art such as Habitat [62] and Li et al. [26]. While the concept of tiling itself is fundamental to GPU programming (e.g., in CUTLASS), its application as the foundational unit for an ML-based performance predictor appears to be a novel contribution. This decomposition simplifies the learning task, as the model only needs to generalize across a smaller set of tile dimensions rather than a vast space of kernel dimensions.

2. Framing the Prediction Target: The most elegant element of novelty is the choice to predict a bounded utilization factor instead of an unbounded latency value. Prior work that attempts to directly predict latency forces the ML model to learn the complex, non-linear physics of the underlying hardware. By instead predicting a value between 0 and 1 that scales a known theoretical bound (the roofline), the authors constrain the problem in a principled way. This is a conceptually significant advance, as it likely accounts for the framework's superior generalization to out-of-distribution GPUs and workloads, which is the primary failure mode of existing methods.

Weaknesses

While the core idea is novel, its foundation and implementation raise questions about the robustness and generality of the contribution.

1. Dependence on Existing Software Conventions: The novelty is tightly coupled to the current implementation paradigm of GPU libraries like cuDNN and CUTLASS, which rely on tiling. The framework's ability to extract tile dimensions is contingent on metadata from tools like the PyTorch Profiler (Section 6.1, page 10). This introduces a degree of brittleness. The proposed novel method is not a first-principles model of GPU execution but rather a model of a specific software ecosystem's execution strategy. A future shift in library design—for example, a move to dynamic or irregular tiling strategies—could potentially invalidate the core assumptions of the framework. The contribution is thus more of a highly effective engineering solution tied to current software, rather than a fundamental and enduring modeling technique.

2. Ambiguity in the "Tile" Abstraction: The paper is convincing for GEMM operators, where the concept of a tile is crisp and well-defined. However, the generalization of this novel abstraction to other operators is less clear. The framework uses five separate MLPs for different operator classes. For element-wise or reduction operators, the notion of a "tile" is conceptually different and less uniform than for matrix multiplication. The paper does not sufficiently detail how the tile-based decomposition is novelly and consistently applied to these other operator types, especially in the context of operator fusion (Section 4.4, page 8), where the control and data flow can become highly complex. The claim of a general "tile-granularity" approach feels somewhat over-extended when its clearest articulation is for GEMM.

3. Compositional Novelty: The contribution is a novel combination of existing concepts. The roofline model [59] is decades-old. MLPs for performance prediction are common. The concept of tile-based execution on GPUs is standard. The novelty here is in the synthesis. While effective, this is an incremental, architectural form of novelty, not a radical new theory of performance modeling.

Questions to Address In Rebuttal

To strengthen the claims of novelty and robustness, the authors should address the following:

1. Generality of the Tile Abstraction: How does the tile-based prediction framework handle operators where the concept of a static, regular "tile" is ill-defined? For instance, how would it model a sparse matrix operation or a complex, hand-optimized kernel from a library like Triton, which may not expose clear tiling metadata in the way that standard cuDNN GEMM kernels do?

2. Future-Proofing the Contribution: The method's dependency on tile metadata from existing profilers is a potential weakness. How would NEUSIGHT adapt if a future version of a major library (e.g., CUTLASS 4.0) fundamentally changes its tiling strategy or ceases to expose this metadata? Is the novel approach extensible to inferring tiling strategies, or is it permanently reliant on reverse-engineering specific library implementations?

3. Operator Fusion Complexity: The description of handling operator fusion in Section 4.4 appears to be an oversimplification (e.g., summing FLOPs and using the first operator's predictor). A fused kernel is not simply the sum of its parts; it has a unique execution profile. Could the authors provide a more detailed example of how the tile-level prediction model, which is the core novel idea, is applied to a non-trivial fused kernel, such as a convolution followed by a bias addition and a ReLU activation? Does a single "tile" prediction still make sense in this context?

Paper: FRUGAL: Efficient and Economic Embedding Model Training with Commodity GPUs Review Form: The Guardian

---

Summary

The paper identifies a critical performance bottleneck when training large-scale embedding models on commodity GPUs: the lack of hardware support for direct peer-to-peer (P2P) communication, which forces all inter-GPU data transfers to be bounced through host memory, incurring significant CPU overhead and communication latency. To address this, the authors propose FRUGAL, a training system built around a "proactively flushing" mechanism. The core idea is to decouple the two halves of an inter-GPU data transfer. The GPU-to-host write is performed proactively and asynchronously in the background, effectively moving it off the critical training path. The host-to-GPU read remains on the critical path but is optimized using Unified Virtual Addressing (UVA). This mechanism is managed by a priority-based algorithm (P2F) that uses the future access step of a parameter as its priority, orchestrated by a custom-designed, two-level concurrent priority queue. The authors claim that FRUGAL significantly improves training throughput on commodity GPUs, reaching performance comparable to datacenter-class GPUs at a fraction of the cost.

---

Strengths

1. Well-Motivated Problem: The paper does an excellent job of motivating the problem. The analysis in Section 2.4, particularly Figure 3, clearly dissects the performance gap between datacenter and commodity GPUs, correctly attributing it to low collective communication bandwidth and CPU involvement overhead. This provides a strong, data-driven foundation for the work.

2. Clever Core Mechanism: The central idea of proactively flushing to exploit the unavoidable host-memory bounce is insightful. Instead of treating this hardware limitation as a pure liability, the authors have engineered a solution that leverages it to hide latency. Decoupling the communication into non-critical (GPU→Host) and critical (Host→GPU) phases is a conceptually strong contribution.

3. Co-design of Data Structure: The design of the two-level priority queue (Section 3.4) is a good example of tailoring a data structure to the specific needs of the algorithm. Recognizing that priorities are bounded integers (training steps) allows for a more efficient implementation (O(1) access) than a generic tree-based heap, which is crucial for reducing background overhead.

---

Weaknesses

My primary concerns with this paper lie in the rigor of its experimental evaluation and the precision of its claims. While the core idea is appealing, the evidence provided is not sufficient to fully substantiate the claims of superiority and cost-effectiveness.

1. Overstated Claims Regarding Datacenter GPU Equivalence: The abstract claims FRUGAL can "achieve similar throughput compared to existing systems on datacenter GPUs." However, the key experiment supporting this (Figure 16, page 12) compares RTX 3090s against NVIDIA A30s. The A30 is a lower-tier, power-efficient datacenter GPU, not a flagship like the A100, which is the standard for high-performance training and features a much more powerful interconnect (NVLink/NVSwitch). This comparison feels carefully selected to yield a favorable result. A true test of equivalence would require a comparison against a system of A100s, where the high-speed interconnect is the dominant factor that FRUGAL aims to circumvent. Without this, the cost-effectiveness claims of "4.0-4.3× improvement" are built on a questionable performance baseline.

2. Potentially Sub-optimal Baseline Implementations: The authors state they "re-implement its multi-GPU cache within PyTorch" (Section 4.1, page 9) to create baselines for HugeCTR and DGL-KE-cached. This is a major methodological concern. The performance of these complex systems is highly dependent on finely-tuned implementations. Comparing FRUGAL against a self-implemented version of a competitor's core feature introduces a clear risk of an un-optimized or "strawman" baseline. This concern is amplified by the scalability results in Figure 15 (page 12), which show the throughput of DGL-KE-cached/HugeCTR decreasing as more GPUs are added. This is highly anomalous behavior for a distributed system and strongly suggests a severe bottleneck in the baseline implementation, rather than an inherent flaw in the caching approach itself. This result undermines the credibility of all experiments where FRUGAL is compared against these baselines.

3. Imprecise and Misleading Terminology: The paper repeatedly makes claims that are technically imprecise. For example, Section 3.1 (page 5) states the goal is to "eliminate GPU collective communication." FRUGAL does not eliminate communication; it restructures it from a single, blocking all-to-all collective into a series of asynchronous point-to-point transfers via host memory. The total volume of data moved between the GPU and host memory system is likely similar, if not greater. This lack of precision weakens the paper's technical arguments. Rigorous systems papers should use precise language.

4. Unexplored Generality and Sensitivity: The proactive flushing mechanism hinges on prefetching the IDs for the next L training steps to determine access priority. The paper sets L=10 by default and does not present a sensitivity analysis on this crucial hyperparameter. For the workloads tested (standard mini-batch training), this prefetching is straightforward. However, the paper does not discuss the limitations of this approach. How would FRUGAL perform in scenarios with dynamic, unpredictable access patterns, or with more complex data sampling strategies where looking ahead is non-trivial? The applicability of the core mechanism seems implicitly tied to a specific, predictable training paradigm.

---

Questions to Address In Rebuttal

The authors must address the following points to strengthen their submission:

1. Justification of Hardware Baseline: Please justify the choice of the A30 GPU as the representative "datacenter GPU" for your headline performance and cost-effectiveness claims. Can you provide data or a well-reasoned argument for why a comparison against a higher-end A100-based system is not necessary to validate your claims?

2. Defense of Re-implemented Baselines: Please provide evidence that your re-implementation of the HugeCTR/DGL-KE caching mechanism is a fair and optimized baseline. Specifically, can you explain the pathological negative scaling behavior seen in Figure 15? Have you compared your re-implementation's performance against the original, vendor-provided HugeCTR framework on the A30 platform to ensure its fidelity?

3. Clarification of "Eliminating Communication": Please either defend the use of the term "eliminate collective communication" with a precise technical definition or revise the paper to more accurately describe the mechanism as a restructuring of communication patterns to hide latency.

4. Sensitivity to Prefetch Depth (L): What is the performance sensitivity of FRUGAL to the prefetch depth L? How does performance degrade as L approaches 1? Please discuss the boundary conditions and limitations of your proactive approach when future data access is not easily predictable.

5. Stall Conditions: The P2F algorithm stalls the training process if the highest priority item in the queue has a priority less than or equal to the current step. Under what conditions (e.g., high parameter contention, slow storage) do the background flushing threads fail to keep up, leading to frequent stalls? An analysis of these stall conditions is necessary to understand the practical robustness of the system.

Reviewer Persona: The Synthesizer

---

Summary

This paper introduces FRUGAL, a training system for large-scale embedding models specifically designed to run on commodity GPUs. The authors correctly identify a critical and growing gap in the field: while commodity GPUs offer excellent cost-performance for computation, they lack the high-speed interconnects (like NVLink and PCIe P2P) found in expensive datacenter-grade GPUs. This hardware limitation cripples existing training systems, which rely on efficient direct GPU-to-GPU communication.

The core contribution of FRUGAL is a paradigm shift in handling communication on this hardware. Instead of GPUs passively waiting to pull needed parameters from peers (a slow process that must bounce through host memory), FRUGAL implements a "proactively flushing" mechanism. Each GPU anticipates future data needs of its peers and pushes its relevant parameter updates to host memory asynchronously. This clever design decouples half of the communication latency from the critical training path. This core idea is supported by a priority-based flushing algorithm (P2F) and a highly-optimized two-level priority queue to ensure correctness and efficiency. The experimental results are strong, demonstrating that FRUGAL not only dramatically boosts performance on commodity hardware but also achieves a cost-effectiveness that is 4.0-4.3x better than datacenter GPUs running existing systems.

Strengths

1. High Significance and Timeliness: The paper addresses an extremely relevant problem. The prohibitive cost of datacenter hardware is a major barrier to entry for academic labs and smaller industrial players. By developing a system that unlocks the potential of affordable, accessible commodity hardware for a critical class of ML models, this work has the potential to democratize research and development in areas like recommendation systems and graph learning.

2. Novel and Insightful Core Idea: The central concept of "proactively flushing" is a genuinely insightful piece of systems co-design. Rather than treating the lack of PCIe P2P as a simple bottleneck to be brute-forced, the authors have re-architected the communication flow to work with the hardware's constraints. Moving the GPU-to-host write operation off the critical path by making it asynchronous and predictive is an elegant solution to a difficult problem. This is a strong example of adapting software architecture to the reality of the underlying hardware.

3. Strong Connection to Broader Systems Concepts: While tailored for a specific problem, the design of FRUGAL resonates with established principles in distributed systems and OS design. The use of asynchrony to hide latency is a classic technique. The P2F algorithm is effectively a form of predictive caching or prefetching, applied here to communication scheduling. By grounding their specific solution in these broader concepts, the authors have created a robust and well-founded system.

4. Excellent Contextualization and Motivation: The background and motivation section (Section 2, pages 3-4) is very well done. The authors clearly explain the architectural differences between datacenter and commodity GPUs (Figure 1), quantify the performance gap with a motivating experiment (Figure 3), and correctly diagnose the root causes as low collective bandwidth and CPU involvement. This sets the stage perfectly for their proposed solution.

Weaknesses

While the work is strong, its focus is necessarily narrow, which brings up some contextual limitations.

1. Intra-Node Focus: The entire design and evaluation of FRUGAL is centered on a single, multi-GPU server. The paper briefly dismisses cross-server distributed training at the end of Section 4.4 (page 12), noting that commodity GPUs are not usually equipped with high-end NICs. While this is true, it is also the most significant limitation of the work. The largest embedding models require multi-node training, and a discussion of how the "proactive flushing" philosophy might (or might not) extend to a networked environment would greatly strengthen the paper's context. Could a similar approach be used with technologies like RDMA to push to remote host memory?

2. Potential Centralized Bottleneck: The controller process, with its global priority queue (PQ), manages the state for all pending updates. While the two-level PQ design is a very clever optimization for this specific use case, it still represents a centralized logical component. The paper does not explore the potential for this controller to become a bottleneck, especially with a higher GPU count or with models that have even more complex access patterns, leading to more frequent PQ operations.

Questions to Address In Rebuttal

1. Could the authors elaborate on the conceptual challenges of extending the FRUGAL philosophy to a multi-node setting? Does the core idea of decoupling communication by pushing to a shared medium (host memory) break down when that medium becomes a slower network fabric, or could it be adapted?

2. The P2F algorithm relies on prefetching future access patterns (the L hyperparameter, page 6). How sensitive is the system's performance to the quality of this prefetching? For example, in graph learning scenarios with dynamic sampling, future accesses might be less predictable. How would FRUGAL's performance be affected in such a scenario?

3. Regarding the controller process: have the authors profiled the CPU utilization of the controller and its flushing threads? At what point (e.g., number of GPUs, update frequency) does this control plane itself begin to consume enough CPU resources to interfere with other parts of the training pipeline, such as data loading and preprocessing?

Reviewer: The Innovator (Novelty Specialist)

---

Summary

This paper presents FRUGAL, a training system for large-scale embedding models specifically designed for commodity GPUs. The authors identify that the primary performance bottleneck on this hardware is the lack of PCIe Peer-to-Peer (P2P) support, forcing all inter-GPU communication to be "bounced" off host memory, which introduces significant latency and CPU overhead.

The core claimed novelty is a "proactively flushing" mechanism, embodied in the Priority-based Proactively Flushing (P2F) algorithm. Instead of GPUs passively waiting to pull required parameters from other GPUs (via the host), each GPU anticipates future parameter needs and proactively pushes its relevant updates to a shared host memory location. The priority of these flush operations is determined by a lookahead into the training data pipeline, specifically, the step number at which a parameter will next be accessed. To manage this efficiently, the authors also propose a custom two-level priority queue data structure. The goal is to hide half of the communication latency (the GPU-to-host write) in a non-critical path, thereby improving end-to-end training throughput while maintaining strict synchronous consistency.

---

Strengths

The primary strength of this paper is the identification and exploitation of a structural property of the commodity GPU communication pattern. The observation that communication must be bounced on host memory (Section 2.4, page 4) is not new, but the idea to re-architect the communication flow from a pull-based model to a predictive push-based model is a novel approach in this specific context.

1. The P2F Algorithm: The core idea of proactively flushing based on future access patterns (Section 3.3, page 6) is a clever synthesis of lookahead scheduling and asynchronous write-back caching. It directly addresses the identified bottleneck and provides a clear mechanism for decoupling the write-phase of communication from the critical training path. This appears to be a genuinely new algorithmic contribution for this problem domain.

2. Maintaining Synchronous Consistency: Many systems achieve performance gains by relaxing consistency models (e.g., stale synchronous parallel). A notable aspect of FRUGAL's proposed novelty is that it aims to hide communication latency without sacrificing the synchronous consistency model, which is critical for model convergence in many commercial applications. The proof provided in Section 3.3 (page 7) is a necessary component to validate this claim.

3. Tailored Data Structure: The two-level priority queue (Section 3.4, page 8) is a well-motivated implementation detail. Recognizing that the priorities are bounded integers (training steps) and designing a data structure with O(1) operations is a significant step beyond naively using a standard binary heap. This demonstrates a thoughtful co-design of the algorithm and its underlying data structures.

---

Weaknesses

My critique focuses on the degree of novelty and the positioning of the work with respect to broader concepts in computer systems. While the specific synthesis of techniques for this problem is new, the constituent components have conceptual precedents that are not fully explored.

1. Conceptual Overlap with Prefetching and Producer-Push Models: At its core, "proactively flushing" is a producer-push data availability model driven by prefetching. The concept of using lookahead into an access stream to move data closer to the consumer before it is requested is the fundamental principle of data prefetching, a well-established technique in memory hierarchies and I/O systems. The paper presents this as a new core idea ("the key idea of FRUGAL is proactively flushing") without sufficiently situating it within this broader class of techniques and clarifying how it differs from, for instance, software prefetching schemes for irregular access patterns.

2. Positioning Relative to Asynchronous Consistency Models: The paper rightly differentiates itself by maintaining synchronous consistency. However, the mechanism of deferring and reordering updates bears a strong resemblance to techniques used to manage updates in asynchronous or semi-synchronous systems (e.g., Parameter Servers). A more thorough discussion is needed to contrast the P2F algorithm not just with synchronous baselines, but also with foundational concepts like Stale Synchronous Parallel (SSP), explaining why hiding latency is a fundamentally different approach than bounding staleness.

3. Novelty of the Priority Queue: The proposed two-level priority queue is described as a custom solution. However, a priority queue for integer priorities over a known range is functionally equivalent to a bucket queue or a calendar queue, which are known data structures. The novelty is therefore not in the data structure itself, but in its specific application and lock-free implementation for this use case. The paper should be more precise in claiming the novelty of the application rather than the structure itself.

---

Questions to Address In Rebuttal

1. Please clarify the conceptual delta between the proposed P2F algorithm and the Stale Synchronous Parallel (SSP) model. While FRUGAL maintains strict consistency, both systems use a lookahead/step-based mechanism to manage parameter updates. How does the goal of hiding latency (FRUGAL) differ from the goal of bounding staleness (SSP) in terms of system design and algorithmic complexity?

2. The premise of the paper is a specific hardware limitation (lack of PCIe P2P) on commodity GPUs. This is an artifact of market segmentation by the hardware vendor. How would the authors position the novelty of FRUGAL if future generations of commodity GPUs were to re-introduce P2P support? Does the core contribution become obsolete, or are there aspects of prioritized flushing that would remain beneficial?

3. Can you elaborate on the relationship between the two-level priority queue (Figure 7, page 8) and the classic "bucket queue" data structure? Acknowledging this connection would help to more accurately frame the contribution as a novel and efficient application of a known data structure pattern to solve a high-concurrency problem in ML systems.

Paper Title: FSMOE: A Flexible and Scalable Training System for Sparse Mixture-of-Experts Models Reviewer: The Guardian

---

Summary

The authors present FSMOE, a training system designed to accelerate the training of sparse Mixture-of-Experts (MoE) models. The system's contributions are threefold: a unified abstraction for MoE modules, a co-scheduling strategy for intra- and inter-node communications, and an adaptive method for partitioning and pipelining gradient communications. The authors claim significant performance improvements over established systems like DeepSpeed-MoE and Tutel, reporting speedups of up to 3.01x on real-world models. The core of their approach relies on an analytical performance model to determine an optimal pipeline degree and a two-step process for partitioning gradients to maximize overlap with computation.

---

Strengths

1. Problem Significance: The paper addresses a well-recognized and critical bottleneck in large-scale model training: the substantial communication overhead introduced by MoE layers. The motivation, as laid out in Section 1 and supported by data in Table 2, is clear and compelling.

2. Sound High-Level Approach: The core ideas of modeling communication/computation costs, co-scheduling different types of network traffic (intra- vs. inter-node), and adaptively partitioning work to maximize overlap are fundamentally sound strategies for performance optimization in distributed systems.

3. Architectural Modularity: The design described in Section 3.1, which breaks the MoE layer into distinct, swappable sub-modules (Gate, Order, Dispatch, etc.), represents good software engineering. This modularity is a prerequisite for the flexibility the system claims.

---

Weaknesses

My primary concerns with this submission center on the robustness of the performance model, the justification for the scheduling heuristics, and the substantiation of the more extreme performance claims, which may mask unfair comparisons or conflated contributions.

1. Oversimplified and Potentially Fragile Performance Model: The entire optimization framework rests on the linear performance models presented in Section 4.1 (page 7). While the authors demonstrate a high R² value for these models against microbenchmarks (Figure 5, page 11), this is insufficient. Microbenchmarks run in isolation and do not capture the complexities of a real, system-wide training run, such as network congestion from competing traffic, PCIe bus contention, or NUMA effects. A model that is predictive in a sterile environment may fail to be accurate under load, rendering the "optimal" pipeline degree r derived from it suboptimal in practice. The paper lacks any validation of the model's predictive power during a full-scale, end-to-end training job.

2. Heuristic-Based Optimization Presented as Principled Solution: The scheduling optimization in Section 4.2 (page 8) is not a true optimization but a classification into one of four predefined cases. This heuristic approach is a significant simplification. The paper provides no justification for why these four cases are exhaustive or how the system behaves at the boundaries between cases, where multiple factors might be equally dominant. A small error in the underlying performance model could easily push the scheduler into the wrong case, leading to a poorly chosen schedule. This raises questions about the robustness of the proposed method.

3. Lack of Rigorous Ablation Study: The system introduces several new techniques simultaneously: online profiling, a new pipeline schedule for intra/inter-node communication, and adaptive gradient partitioning. However, the evaluation does not properly isolate the contribution of each component. For instance, the "FSMoE-No-IIO" variant in Figure 6 is a good start, but it is not a full ablation. How much of the benefit comes only from the adaptive gradient partitioning (Section 5) versus the improved forward/backward pass pipelining (Section 4)? Without this breakdown, it is impossible to assess the true value of each proposed technique. The complexity of the full FSMOE system may not be justified if a single, simpler component is responsible for most of the gains.

4. Confounding Variables in Performance Comparisons:

   • The 3.01x Speedup Claim: The 3.01x speedup over DeepSpeed-MoE for GPT-XL on Testbed A (Figure 6a, page 12) is an extraordinary claim that requires extraordinary evidence. A detailed analysis is missing. Is this a case where the specific model configuration exposes a known pathological weakness in DeepSpeed-MoE's scheduler? Is the baseline implementation properly tuned? Without a root-cause analysis explaining why the baseline is so slow in this specific scenario, this result appears to be an outlier at best and a case of cherry-picking at worst. The more modest ~1.19x speedup over Tutel feels more representative, and the paper should focus on justifying that.
   • Gating Function Performance: Table 6 (page 13) shows that FSMOE's implementations of various gating functions are faster than DeepSpeed-MoE's. This is presented as a strength of the FSMOE framework, but it is unclear if this gain is due to the novel scheduling system or simply a more optimized CUDA kernel implementation for the gating functions themselves. If the latter, it is a valid but separate contribution that should not be conflated with the paper's primary claims about task scheduling.

---

Questions to Address In Rebuttal

1. The performance model in Section 4.1 is validated against isolated microbenchmarks. Can you provide evidence of your model's predictive accuracy for communication and computation primitives during a full end-to-end training run, where network and system resources are contended? How robust is the choice of the optimal pipeline degree r to inaccuracies in this model?

2. Please justify the heuristic classification into four cases in your scheduling algorithm (Section 4.2). Provide analysis on the sensitivity of this classification. What happens if a workload lies on the decision boundary between two cases (e.g., when inter-node communication time is nearly equal to expert computation time)?

3. Regarding the 3.01x speedup over DeepSpeed-MoE shown in Figure 6a, please provide a detailed performance breakdown (e.g., via a timeline visualization or profiling data) for both FSMOE and the baseline. What specific operations account for the massive performance difference in this configuration?

4. Could you provide a more comprehensive ablation study that isolates the performance gains from: (a) the adaptive gradient partitioning method alone, and (b) the intra-/inter-node communication pipelining alone? This is crucial to understanding the relative importance of your contributions.

5. For the results in Table 6, can you confirm whether the performance gains on different gating functions are a result of your scheduling framework or due to superior, low-level kernel implementations compared to the baseline? An experiment running these gating functions in isolation, outside of any scheduling framework, would clarify this point.

Paper Title: FSMoE: A Flexible and Scalable Training System for Sparse Mixture-of-Experts Models Reviewer Persona: The Synthesizer (Contextual Analyst)

---

Summary

This paper introduces FSMoE, a training system for sparse Mixture-of-Experts (MoE) models that tackles the critical challenge of communication overhead in complex hybrid-parallel settings. The authors identify that existing systems either lack flexibility or fail to optimally schedule the multiple, often competing, communication patterns that arise from combining Data, Model, Expert, and Expert-Sharding parallelism (DP, MP, EP, ESP).

The core contribution is a holistic scheduling framework that co-optimizes these communication patterns. This is achieved through three key techniques: 1. A modular abstraction of the MoE layer, enabling flexibility and profiling of different components (e.g., routing functions). 2. A novel scheduling algorithm that pipelines inter-node communication (from EP's AlltoAll) with intra-node communication (from ESP's collectives) and expert computation. This is guided by an analytical performance model that intelligently selects the optimal pipeline depth. 3. An adaptive gradient partitioning method that overlaps the DP's Gradient-AllReduce communication with the backward pass of the MoE layer, treating it as a co-design problem with the primary MoE scheduling.

Experimental results on two GPU clusters demonstrate significant speedups, outperforming state-of-the-art systems like DeepSpeed-MoE and Tutel by 1.18x-1.22x on configured layers and up to 3.01x on real-world models.

Strengths

1. Excellent Problem Contextualization and Significance: The paper does a superb job of situating itself within the broader landscape of large-scale model training. The motivation is clear and compelling: as MoE models grow, the interplay between different parallelism strategies creates a complex scheduling problem where communication is the dominant bottleneck (as shown in Table 2, page 5). This work directly addresses a timely, expensive, and high-impact problem faced by nearly everyone training frontier models.

2. Holistic, Multi-Layered Optimization: The most significant strength of this work is its recognition that MoE training is not a "one-bottleneck" problem. Previous works like Tutel/PipeMoE [17, 42] focused primarily on overlapping the main AlltoAll collective with expert computation. FSMoE elevates this by considering the system holistically. It models and co-schedules three distinct, potentially conflicting, communication patterns: the intra-node ESP collectives, the inter-node EP AlltoAll, and the inter-node DP AllReduce. This multi-layered view is a natural and important evolution in the field. The adaptive gradient partitioning (Section 5, page 9) is a particularly insightful piece of this co-design.

3. Principled, Model-Driven Approach: The scheduling solution is not based on simple heuristics but on a principled, model-driven optimization. By creating linear performance models for communication and computation (Section 4.1, page 7) and categorizing the problem space into four distinct cases (Figure 4, page 7), the authors transform a complex scheduling challenge into a series of solvable optimization problems. This systematic approach is a hallmark of strong systems research and adds significant credibility to the results.

4. Strong Empirical Validation: The evaluation is thorough and convincing. The authors not only show significant end-to-end speedups against strong baselines on real-world models (Figure 6, page 12) but also include what amounts to an ablation study in their comparisons (e.g., comparing FSMoE vs. FSMoE-No-IIO vs. Tutel-Improved). This clearly isolates and validates the performance gains from their specific contributions, particularly the benefit of co-scheduling inter- and intra-node communication.

Weaknesses

While the paper is strong, there are opportunities to further contextualize and strengthen its claims.

1. Implicit Assumptions in the Core Scheduling Model: The core scheduling optimization (Section 4, pages 6-9) is developed for the "common scenario" where the MP and ESP groups are aligned with the number of GPUs per node. While this is a very practical and common topology, it simplifies the problem by creating a clean separation between fast intra-node (NVLink) and slower inter-node (InfiniBand/Ethernet) communication. The work would be more broadly impactful if it discussed how its principles might extend to more heterogeneous or irregular topologies, where the distinction between "intra" and "inter" is less clear-cut. This is not a flaw in the current work, but a question of its generality.

2. Positioning as Synthesis vs. Pure Novelty: The paper builds intelligently on a chain of prior work. The idea of pipelining by splitting the input tensor was explored in PipeMoE [42], and the problem of contention between AllReduce and AlltoAll was a central theme in Lina [24]. FSMoE’s key contribution is synthesizing these ideas into a more general and adaptive framework. The paper could strengthen its narrative by more explicitly framing itself as a unifying work that generalizes previous point solutions into a more comprehensive, model-driven scheduler, rather than implying each component is entirely novel.

Questions to Address In Rebuttal

1. The core scheduling algorithm is predicated on the assumption that N_ESP = N_MP = GPUs_per_node. Could the authors comment on how their performance models and scheduling principles would adapt to scenarios where this is not the case? For example, in a system with very high inter-node bandwidth, would the sharp distinction between intra- and inter-node scheduling still be the optimal approach?

2. The adaptive gradient partitioning in Section 5 is a compelling idea that improves upon prior work like Lina [24], which uses fixed-size chunks. Could you quantify the benefit of this "adaptive" partitioning? For instance, how much does the optimal amount of partitioned gradient vary across different layers or model configurations, and what is the performance cost of using a fixed, non-adaptive scheme in those cases?

3. The online profiling and model-fitting step is critical to the system's adaptivity. What is the one-time cost of this profiling on a new cluster, and how sensitive is the scheduler's final performance to minor inaccuracies in the fitted performance models (α and β values)? A brief discussion on the robustness of the system would be valuable.

Reviewer: The Innovator (Novelty Specialist)

---

Summary

This paper introduces FSMoE, a training system for sparse Mixture-of-Experts (MoE) models designed to optimize performance by improving task scheduling. The authors identify communication, particularly the interplay between intra-node (e.g., ESP-AllGather) and inter-node (e.g., AlltoAll, Gradient-AllReduce) collectives, as the primary bottleneck.

The core claims of novelty rest on three techniques: 1. A modular software abstraction for MoE components. 2. A co-scheduling methodology that pipelines inter-node and intra-node communications with computation, supported by an analytical model to determine the optimal pipeline degree. 3. An adaptive gradient partitioning method to maximize the overlap of the Gradient-AllReduce collective with other operations in the backward pass.

The paper presents a system that models the performance of these constituent operations and uses these models to solve optimization problems to derive a near-optimal execution schedule.

Strengths (Novelty-focused)

The primary novel contributions of this work lie in its sophisticated, model-driven scheduling algorithms, which represent a significant step beyond prior heuristic or fixed-policy approaches.

1. Adaptive Gradient Partitioning: The most significant novel idea is the adaptive gradient partitioning scheme detailed in Section 5. Prior work, such as Lina [24], has explored partitioning the gradient update to overlap AllReduce with other operations. However, Lina [24] uses a fixed chunk size, which is a static heuristic. The method proposed here is fundamentally more advanced. The two-step process—(1) calculating the available "overlappable time" from other layers and slicing the gradient to precisely fill these gaps, and (2) optimizing the assignment of the remaining gradient to the MoE layers—is a genuinely new algorithm for this domain. This adaptivity, based on profiled performance of the specific model and hardware, is the key innovation.

2. Analytical Pipeline Optimization: The methodology in Section 4.2 for optimizing the pipeline degree r is also novel. While pipelining communication and computation is a well-established technique (e.g., PipeMoE [42]), this work formalizes the optimization. By classifying the scheduling problem into four distinct cases based on the dominant bottleneck (inter-node communication, expert computation, etc.) and formulating a constrained optimization problem for each, the authors provide a principled way to derive the optimal pipeline depth. The insight to calculate separate optimal degrees for the forward and backward passes (Section 4.4) is a logical but important extension that distinguishes it from systems that use a single, globally-set degree.

3. Holistic Co-Design: The strength of the work lies not just in these two ideas in isolation, but in their co-design. The pipeline scheduling of Section 4 creates the temporal "slots" and opportunities for overlap, which the adaptive gradient partitioning algorithm of Section 5 then intelligently fills. This holistic view of the entire backward pass, treating both the MoE-specific collectives and the standard DP gradient collectives as part of a single, global scheduling problem, is a novel perspective.

Weaknesses (Novelty-focused)

While the core scheduling algorithms are novel, some of the paper's contributions are framed in a way that overstates their novelty.

1. "Unified Abstraction" is an Engineering Contribution, Not a Research Novelty: The modular framework described in Section 3.1, with its distinct sub-modules (Gate, Order, etc.) and hooks, is an example of good software engineering. However, it is not a novel research concept. Such abstractions are standard practice in designing flexible software systems and are conceptually similar to patterns found in modern deep learning frameworks. While this framework enables the novel scheduling work, it should be positioned as an implementation detail rather than a primary novel technique.

2. Insufficient Differentiation from Conceptually Similar Prior Art: The paper's novelty would be clearer with a more direct and detailed comparison to conceptually overlapping work in the text.

   • Lina [24]: The experimental comparison is present, but the Related Work or methodology sections should explicitly detail why FSMoE's adaptive, model-driven partitioning is superior to Lina's fixed-chunk partitioning from a conceptual standpoint.
   • Centauri [5]: This work also focuses on communication partitioning to enable overlap. While the domain (general LLMs vs. MoE-specific) and mechanisms differ, the high-level goal is identical. The authors should include a discussion that contrasts their MoE-centric, whole-pass optimization with Centauri's approach to firmly establish their unique contribution.
   • The general concept of overlapping communication and computation is, of course, not new (e.g., T3 [34], CoCoNet [18]). The paper's claims must be precisely focused on the specific mechanisms for scheduling in the MoE context.

Questions to Address In Rebuttal

1. Could the authors please elaborate on the conceptual delta between their adaptive gradient partitioning scheme (Section 5) and the methods proposed in Lina [24] and Centauri [5]? Specifically, what fundamental limitations in those prior works does your adaptive, two-step optimization model overcome?

2. The four-case analytical model in Section 4.2 is presented as the basis for optimizing the pipeline degree. Is this model exhaustive? Can you discuss potential scenarios, perhaps involving heterogeneous hardware or unconventional network topologies, where these four cases might not adequately capture the performance bottlenecks, and how your system would adapt?

3. The proposed scheduling relies on solving a set of constrained optimization problems. While the one-time cost is acceptable, this ties the system to the specific performance characteristics of the operations modeled (AlltoAll, AllGather, GEMM, etc.). How robust is this novel scheduling framework to the introduction of entirely new types of operations or parallelism dimensions (e.g., sequence parallelism)? Would it require a complete re-derivation of the underlying analytical models, or can the framework be extended compositionally?

Paper Title: Fusion: An Analytics Object Store Optimized for Query Pushdown Reviewer Persona: The Guardian (Adversarial Skeptic)

---

Summary

The paper presents Fusion, an object store that aims to optimize query pushdown performance on erasure-coded data. The central thesis is that existing systems are inefficient because the fixed-size block abstraction of erasure coding causes fragmentation of semantic data units (e.g., Parquet column chunks). Fusion's primary contribution is a technique called File-Format-Aware Coding (FAC), which co-designs erasure coding and file layout. FAC uses a greedy bin-packing algorithm to create variable-sized data blocks that align with column chunk boundaries, thereby preventing splits. This is coupled with a simple cost model to selectively push down parts of a query based on data compressibility and query selectivity. The authors claim significant median and tail latency improvements over a baseline system with modest storage overhead.

Strengths

1. Problem Formulation: The paper correctly identifies a significant and timely problem. The impedance mismatch between the logical structure of analytics file formats and the physical block layout of erasure-coded storage is a well-known, practical bottleneck for query pushdown. The motivation is clear and well-articulated.
2. Conceptual Approach: The core idea of making the storage layer aware of the file format's internal structure is sound. Moving away from a fixed-block-size abstraction for erasure coding to prevent fragmentation of computable units is a logical and promising direction.
3. Clarity: The paper is well-written, and the core concepts of FAC and the stripe construction algorithm are explained clearly, particularly with the aid of figures like Figure 8 and Figure 9.

Weaknesses

My assessment reveals several critical weaknesses in the methodology and evaluation that call the paper's central claims into question.

1. Oversimplified and Unvalidated Cost Model: The adaptive pushdown mechanism hinges on a cost model described as selectivity × compressibility < 1 (Section 4.3, page 8). This is not a cost model; it is a simple heuristic. It completely ignores fundamental factors that dominate performance in a distributed system:

   • Network Bandwidth: The model implicitly assumes infinite, zero-latency network, which is unrealistic. The decision to pushdown versus fetch should explicitly model the transfer time (size / bandwidth).
   • CPU Cost: Decompression and predicate evaluation on storage nodes consume CPU. The model ignores this cost, which can be significant for complex predicates or CPU-intensive decompression algorithms.
   • Disk I/O: The model does not account for the I/O cost of reading the column chunk from disk on the storage node. This heuristic is presented without any validation or sensitivity analysis. It is highly probable that in a real-world scenario with network congestion or CPU-bound storage nodes, this simplistic model would make incorrect, performance-degrading decisions.

2. The Baseline is a Potential Strawman: The authors state they "implement a baseline system representative of state-of-the-art systems such as MinIO and Ceph" (Section 6, page 10). A self-implemented baseline is a major methodological red flag. Production systems like MinIO and Ceph have years of performance engineering behind them. The baseline's described behavior—assembling the Parquet object on a coordinator node before execution—seems particularly naive. An optimized baseline might perform parallel fetches of the required chunk fragments to the coordinator and begin processing as data streams in, overlapping network I/O with computation. Without a direct comparison to an actual, tuned production system, or at least a much more rigorous justification of the baseline's design, the reported performance gains are suspect. The entire evaluation may be predicated on outperforming an unoptimized implementation.

3. Misleading Presentation of Performance Gains: The abstract and introduction prominently feature dramatic latency improvements: "improves median and tail latency by 64% and 81%, respectively". However, a close reading of the evaluation reveals these numbers are cherry-picked from the best-case results of a microbenchmark run against individual columns of the TPC-H lineitem table (Section 6.1, page 11, Figure 13). The evaluation on more realistic, multi-column "real-world SQL queries" shows much more modest gains of "up to 40% and 48% respectively" (Section 6.2, page 12, Figure 15). The abstract should report the results from the more holistic and realistic workload, not the best-case microbenchmark. This presentation feels like a bait-and-switch.

4. Insufficient Evaluation of the Stripe Construction Algorithm: The FAC stripe construction algorithm (Algorithm 1) is a greedy heuristic. The paper claims it incurs only a "1.2% storage overhead compared to the optimal." However, the evidence for this is thin.

   • The primary evaluation of its overhead is performed on a synthetic dataset with a Zipf distribution (Figure 16a). The paper itself shows that real-world column chunk size distributions are varied and do not necessarily follow a Zipfian pattern (Figure 4c). There is no analysis showing the heuristic's robustness across these more realistic distributions.
   • The paper acknowledges a worst-case storage overhead of (n-k) (Section 4.2, page 8), which is equivalent to replication and would negate the primary benefit of erasure coding. This catastrophic case is mentioned but not explored. Under what real-world conditions might this occur? The evaluation seems to conveniently ignore distributions that might stress the heuristic.

Questions to Address In Rebuttal

The authors must address the following points to substantiate their claims:

1. Cost Model Justification: Please provide a rigorous justification for the selectivity × compressibility < 1 heuristic. How would this model change if network bandwidth, CPU decompression costs, and disk I/O latency were explicitly included? Present empirical data showing that your simplification does not lead to suboptimal pushdown decisions across a range of hardware configurations and query types.
2. Baseline Fidelity: The baseline system is a custom implementation. Can you provide evidence that its performance is representative of a production system like MinIO or Ceph running on your testbed? Specifically, does the baseline's data reassembly strategy represent the state-of-the-art, or is it a simplified strawman designed to be easily outperformed?
3. Discrepancy in Reported Gains: The abstract claims up to 81% tail latency improvement. This figure is from a microbenchmark on a single, highly favorable column. The more realistic, end-to-end query evaluation shows a maximum of 48% improvement. Please clarify this discrepancy and justify why the headline claim in the abstract is based on the microbenchmark rather than the more representative workload.
4. Robustness of the FAC Heuristic: The FAC stripe construction algorithm's overhead is primarily evaluated against a synthetic Zipf distribution. Given the diverse, non-Zipfian distributions shown for real-world datasets in Figure 4c, what evidence can you provide that the algorithm's near-optimal performance (1.2% overhead) holds for these more complex data distributions? Furthermore, please characterize the conditions under which the algorithm's performance degrades towards its worst-case overhead.

Paper Title: Fusion: An Analytics Object Store Optimized for Query Pushdown Reviewer: The Synthesizer (Contextual Analyst)

---

Summary

This paper presents Fusion, an analytics object store that addresses a critical performance bottleneck in modern cloud data architectures: the inefficiency of query pushdown on erasure-coded data. The central problem identified is that conventional object stores treat complex, structured analytics files (like Parquet) as opaque blobs. When applying erasure coding, they slice these blobs into fixed-sized blocks, which frequently splits the file's "smallest computable units" (e.g., Parquet column chunks) across multiple storage nodes. This fragmentation forces a costly data reassembly step over the network before any pushed-down computation can occur, negating many of the benefits of pushdown.

The core contribution of Fusion is a novel technique called File-Format-Aware Coding (FAC). Instead of using fixed-sized storage blocks, FAC co-designs the erasure coding process with the internal structure of the analytics file. It intelligently groups column chunks into variable-sized data blocks, ensuring that no single computable unit is ever split. To manage the potential storage overhead of this variable-size approach, the authors frame the problem as a novel variant of bin packing and propose an efficient heuristic algorithm. This architectural change allows Fusion to perform granular, cost-aware query pushdown directly on intact column chunks at the storage nodes, dramatically reducing network traffic and latency for selective queries.

---

Strengths

1. Fundamental Insight and Novel Co-Design: The paper's primary strength lies in its clear and powerful core idea: breaking the abstraction barrier between the storage system's erasure coding layer and the analytics file format layer. This is a significant departure from the siloed design of current systems. By making the storage layer "semantically aware" of the data it's protecting, Fusion addresses the root cause of the performance issue, rather than applying a patch. This cross-layer optimization is elegant and has the potential to influence the design of future storage systems for analytics.

2. Excellent Problem Motivation: The authors do a superb job of contextualizing their work. The introduction clearly explains the architectural shift to disaggregated storage in the cloud and how this has created the very problem Fusion aims to solve. The motivating example and the empirical data in Section 3 (specifically Figure 4) convincingly demonstrate that column chunk splitting is a real and frequent problem, and that the resulting network reassembly costs are substantial. This sets the stage perfectly for the proposed solution.

3. Holistic and Practical System Design: Fusion is not just a single trick; it's a well-reasoned system. The authors correctly identify that using variable-sized blocks for erasure coding introduces the challenge of storage overhead. Their formulation of this as a bin-packing problem is insightful, and the development of a lightweight heuristic (Algorithm 1) over an impractical ILP solver shows a keen focus on practical implementation. Furthermore, the inclusion of a fine-grained, adaptive cost model for query pushdown (Section 4.3) adds a layer of intelligence, recognizing that pushdown is not a universal panacea and depends on query selectivity and data compressibility. This demonstrates a mature understanding of the problem space.

4. Significant Performance Improvements: The experimental results are compelling, showing significant reductions in median and especially tail latency (up to 81% on TPC-H) on a key metric for interactive analytics. The detailed breakdowns (e.g., Figure 13c) clearly attribute these gains to the reduction in network overhead, directly validating the paper's central hypothesis.

---

Weaknesses

While the core idea is strong, a broader contextual analysis raises some points that could be strengthened. These are less flaws in the work itself and more opportunities to explore the implications of its design.

1. The Trade-offs of Tight Coupling: The primary contribution of Fusion—the co-design—is also a source of potential weakness. By making the storage layer aware of the Parquet file format, the system introduces a tight coupling between layers that were previously independent. This has software engineering and maintenance implications. How does Fusion handle evolving file format specifications (e.g., new Parquet versions)? Does the storage system now require specific modules or plugins for every analytics format it wishes to optimize? A more explicit discussion of this classic "performance vs. abstraction" trade-off would enrich the paper.

2. Generalizability Beyond Columnar Formats: The work is presented almost entirely in the context of PAX-based columnar formats (Parquet, ORC). This is a massive and important domain, but the underlying principle of preserving "computable units" could be far more general. Could this approach apply to other data types where fragmentation is costly? For example, preserving individual frames or GOPs in video files for in-storage transcoding, or preserving logical records in scientific data formats like HDF5. Exploring the applicability of the FAC concept to other domains would help frame it as a more foundational storage primitive.

3. Simplicity of the Heuristic: The proposed stripe construction algorithm is simple and fast, which is a virtue. However, its greedy, "largest-first" nature may be suboptimal for certain pathological, yet potentially real, distributions of column chunk sizes. The evaluation shows it performs well on the tested datasets, but a brief discussion of its theoretical limitations or worst-case behavior (even if mitigated by the storage overhead threshold) would provide a more complete picture.

---

Questions to Address In Rebuttal

1. Could you please comment on the software engineering implications of the tight coupling introduced by FAC? How do you envision a system like Fusion supporting a growing and evolving ecosystem of analytics file formats without becoming a maintenance bottleneck?

2. The concept of preserving "computable units" seems broadly applicable. Beyond Parquet and ORC, what other data formats or application domains do you believe could significantly benefit from a file-format-aware erasure coding approach like the one you propose?

3. Your stripe construction heuristic is designed for speed. Could you discuss the scenarios or chunk size distributions where this heuristic might perform poorly compared to the optimal solution, and elaborate on how the system's configurable storage overhead threshold serves as a practical safeguard against this?

Reviewer: The Innovator (Novelty Specialist)

Summary

The paper presents Fusion, an analytics object store designed to optimize query pushdown on erasure-coded data. The central problem identified is the mismatch between fixed-size blocks used by conventional erasure coding schemes and the variable-sized, semantically meaningful "computable units" (e.g., column chunks) in analytics file formats like Parquet. This mismatch leads to fragmentation of computable units across storage nodes, necessitating costly data reassembly before query predicates can be applied, thereby nullifying many of the benefits of pushdown.

The authors' core proposed solution is a technique called File-Format-Aware Coding (FAC). FAC co-designs the erasure coding layer with the file format by constructing stripes with variable-sized data blocks that align perfectly with the boundaries of column chunks. This prevents fragmentation. To manage the storage overhead that naively using variable-sized blocks would introduce, FAC employs a stripe construction algorithm based on a novel formulation of the bin packing problem. A secondary contribution is a cost model to adaptively enable or disable pushdown at the column-chunk level based on query selectivity and data compressibility.

---

Strengths

The primary strength of this paper lies in its novel and elegant solution to a well-defined and increasingly important problem. My evaluation of the paper's novelty is as follows:

1. Novel Core Mechanism (FAC): The core idea of using variable-sized blocks within a single erasure code stripe to respect the semantic boundaries of application data units is genuinely novel in the context of modern object stores. While the general concept of "content-aware" or "application-aware" storage is not new, its specific application to solve the fragmentation problem in erasure coding for analytics formats is a significant and previously unexplored design point. It directly addresses the shortcomings of the closest prior art, which relies on inefficient padding to achieve alignment [36].

2. Novel Problem Formulation: The authors correctly identify that minimizing storage overhead in their variable-block-size scheme is equivalent to minimizing the sum of the sizes of the largest block in each stripe. They formalize this as a variant of the bin packing problem. To my knowledge, this specific objective function—minimizing the sum of maximums across multiple bin sets—is a new formulation not found in classic bin packing or scheduling literature (which typically focuses on minimizing the number of bins or the makespan, i.e., the max of maxes). This demonstrates a deep understanding of the problem's theoretical underpinnings.

3. Elegant and Practical Heuristic: The paper presents a simple, fast, and effective greedy heuristic (Algorithm 1) for their NP-complete stripe construction problem. The design principles of the algorithm (e.g., placing the largest remaining chunks first to bound the stripe's overhead) are sound and well-justified. The fact that its runtime complexity is negligible on the critical write path makes the entire approach practical.

---

Weaknesses

While the primary contribution is strong, the novelty of the secondary contributions is less pronounced.

1. Limited Novelty in Adaptive Pushdown: The cost model for adaptive query pushdown presented in Section 4.3 (selectivity × compressibility < 1) is an expression of a well-understood principle in distributed query optimization. The fundamental trade-off—comparing the cost of transferring compressed source data versus transferring larger, uncompressed results after remote computation—is a classic one. Many distributed database optimizers perform a similar, albeit often more complex, cost analysis. The novelty here is not in the concept itself, but rather in its straightforward application at a fine-grained, per-column-chunk level, which is itself only made possible by the primary FAC innovation. The paper should be more precise in positioning this as an essential engineering component for system robustness, rather than a fundamental research contribution.

2. Unexplored Generalizability: The FAC concept is presented almost exclusively in the context of Parquet and its PAX layout. While Parquet and ORC are dominant, the novelty would be strengthened by a discussion of how the core idea could be generalized. For instance, how would FAC apply to log-structured data, graph formats with variable-sized node/edge lists, or other structured formats that may have different definitions of a "computable unit"? The paper misses an opportunity to frame FAC as a more general principle for co-designing storage redundancy with data semantics.

---

Questions to Address In Rebuttal

1. On the Bin Packing Formulation's Novelty: The paper claims the bin packing formulation is a new variant. Can you please elaborate on the relationship between your objective function (minimizing the sum of max-sized items per bin set) and other, potentially related problems in the literature, such as variable-sized bin packing or scheduling problems? A clearer demarcation from the closest theoretical prior art would strengthen this claim.

2. On the Adaptive Pushdown Contribution: I argue that the cost model for adaptive pushdown is an application of a known principle. Could you clarify what you see as the core novel insight in this mechanism, beyond its application at the granularity enabled by FAC? To help isolate its contribution, could you provide data on the performance degradation if a non-adaptive "always pushdown" policy were used with FAC, particularly for the worst-case queries highlighted in Figure 10b?

3. On the "All-or-Nothing" Bin Sealing: Algorithm 1 appears to seal the first bin (containing the largest chunk) and set its size as the capacity for all other bins in that stripe. This seems to imply that no chunk larger than the smallest item in the first bin can be placed in any other bin in the stripe. Is this interpretation correct? If so, does this greedy choice ever lead to significantly sub-optimal packing for future stripes by "trapping" medium-sized chunks that could have been packed more efficiently?

Paper Title: GRAPHPIPE: Improving Performance and Scalability of DNN Training with Graph Pipeline Parallelism Reviewer: The Guardian (Adversarial Skeptic)

---

Summary

The paper introduces 'Graph Pipeline Parallelism' (GPP), a scheme for parallel DNN training that partitions the model into a directed acyclic graph (DAG) of stages, in contrast to the linear sequence of stages used in existing 'Sequential Pipeline Parallelism' (SPP) systems. The authors argue that GPP can better exploit the inherent parallelism in multi-branch DNNs, leading to shorter effective pipeline depths, reduced memory for activations, and consequently, higher throughput. They present GRAPHPIPE, a system that implements GPP through a series-parallel decomposition and dynamic programming-based partitioner and a static micro-batch scheduler. Evaluation on three multi-branch models shows speedups of up to 1.6x over PipeDream and Piper, and a claimed 9-21x reduction in strategy search time.

Strengths

1. Sound Conceptual Foundation: The core premise—that preserving a DNN's graph topology is a more general and potentially more efficient parallelization strategy than enforcing a linear sequence of stages—is logically sound. The critique of linearization losing parallel opportunities is valid in principle.
2. Complete System Implementation: The authors have gone beyond simulation and implemented a full system, including a partitioner, scheduler, and a distributed runtime integrated with FlexFlow. This demonstrates a significant engineering effort and allows for real-world performance measurements.
3. Problem Formulation: The formal problem definition in Section 3 is well-structured, correctly identifying the min-max optimization objective and generalizing the pipeline stage definition to accommodate graphical dependencies and per-stage micro-batch sizes.

Weaknesses

My primary concerns with this work stem from a potentially biased evaluation scope and strong, potentially unsubstantiated, claims based on a heavily heuristic-driven search algorithm.

1. The "SPP" Straw Man: The entire motivation for GPP hinges on the argument that SPP systems create "imaginary linear dependencies" (Figure 2, page 3). This depiction feels overly simplified. It is not rigorously established that state-of-the-art SPP optimizers (like PipeDream's) are fundamentally incapable of finding partitions that group independent operators into the same stage or adjacent stages to maintain parallelism. The paper lacks a detailed analysis of why an existing SPP optimizer fails, instead presenting a caricature of SPP that perfectly motivates GPP. The central premise requires stronger evidence that this is an unavoidable flaw of the SPP model, not just a weakness of a specific implementation.

2. Narrow and Biased Evaluation: The experimental evaluation (Section 7, Figure 6) is exclusively focused on multi-branch DNNs (MMT, DLRM, CANDLE-Uno). These models are the ideal use case for GPP and are practically guaranteed to show its benefits. This is a critical omission. A method that claims to "generalize existing sequential pipeline parallelism" (Abstract, page 1) must be validated on the very workloads it claims to generalize: large, predominantly sequential models (e.g., a standard GPT-style Transformer). While Appendix A.3 presents a table showing comparable performance on a sequential Transformer, this is insufficient. This result should be in the main body, accompanied by an analysis of the partitioning decisions, memory usage, and pipeline schedules. Without this, the work appears to be a specialized solution for multi-branch models rather than a true, general improvement.

3. Conflation of Search Time and Solution Quality: The paper touts a 9-21x search time reduction as a major contribution (Section 7.2, Table 1). However, this is a direct result of the strong heuristic used: series-parallel decomposition. This heuristic massively prunes the search space. The authors fail to demonstrate that this heavily restricted search space still contains the optimal GPP solution, or that the solutions it finds are consistently superior to those found by the more exhaustive search of an SPP system over a linearized graph. The faster search time is meaningless if it comes at the cost of finding a globally superior solution that may be non-series-parallel. The authors present speed as an unqualified good, without analyzing the trade-off with optimality.

4. Oversimplified Analysis of Performance Gains: The mechanistic explanation for the performance gains relies heavily on a case study of a synthetic Transformer model (Section 7.5, Figure 9). The conclusion that the 20% gain is a neat 10% from reduced pipeline bubbles and 10% from larger micro-batches is convenient but may not be generalizable. The paper fails to provide a similar, detailed breakdown for the real-world models in the main evaluation. The ablation study (Figure 8) is a good first step, but it simply separates "Parallel" from "GraphPipe" (which includes larger micro-batches), lacking the fine-grained analysis of the case study. This reliance on a synthetic model to explain the core results is a significant weakness.

5. Lack of Robustness Analysis for the Partitioner: The partitioner described in Algorithm 1 is a complex system of nested heuristics (binary search on TPS, DP, series-parallel decomposition). The paper presents this algorithm without any sensitivity analysis. How does the solution quality change with different initial bounds for the binary search (MAXTPS)? How does the enumeration of "candidate schedule configurations" (Line 15) scale, and how is this set C constructed and bounded? The paper lacks the rigorous analysis required to convince the reader that this complex heuristic is robust and not overtuned to the specific models tested.

Questions to Address In Rebuttal

1. Regarding the evaluation scope: Why were no large, primarily sequential models (e.g., a standard decoder-only Transformer) included in the main evaluation (Figure 6)? The appendix (A.3) shows comparable performance, but the lack of analysis in the main body weakens the claim that GPP is a true generalization of SPP. Please provide a detailed analysis of GPP's performance and partitioning decisions on such a model.

2. Regarding the search space: The significant reduction in search time is attributed to the series-parallel decomposition heuristic. Can the authors provide evidence or a formal argument that this restriction does not prevent the partitioner from finding solutions that are superior to those discoverable by more exhaustive (albeit slower) SPP methods? Is it possible that the optimal graph partition is not series-parallel, and is thus missed by your algorithm?

3. Regarding the comparison to SPP: The paper frames SPP as creating "imaginary linear dependencies" (Figure 2). Could you clarify whether this is an inherent, fundamental limitation of the SPP formulation itself, or a limitation of specific implementations? Is it not possible for a sophisticated SPP partitioner to identify and co-locate independent operators to avoid such dependencies within its linear sequence?

4. Regarding the source of performance gains: The case study (Section 7.5) on a synthetic model attributes the 20% gain to a 50/50 split between reduced pipeline bubble and improved compute efficiency. Please provide a similar breakdown for the real-world models (MMT, DLRM, CANDLE-Uno) evaluated in Figure 6. Does this 50/50 split generalize, or is the dominant factor model-dependent?

Reviewer: The Synthesizer (Contextual Analyst)

---

Summary

This paper introduces Graph Pipeline Parallelism (GPP), a compelling and natural generalization of traditional sequential pipeline parallelism (SPP) for training large-scale Deep Neural Networks (DNNs). The central thesis is that existing pipeline parallelism methods, by linearizing a DNN's computation graph, fail to exploit the inherent parallelism present in modern multi-branch architectures (e.g., multi-modal models, recommendation systems).

GPP addresses this by partitioning a DNN into a Directed Acyclic Graph (DAG) of stages, preserving the model's intrinsic topology. This approach enables the concurrent execution of computationally independent branches, leading to two key benefits: 1) a reduction in the effective pipeline depth, which shrinks the "pipeline bubble" and lowers activation memory requirements, and 2) improved GPU utilization by allowing for larger micro-batch sizes due to the freed memory. The authors embody this concept in a system called GRAPHPIPE, which includes a topology-aware partitioner and a static micro-batch scheduler. Through extensive experiments on relevant multi-branch models, they demonstrate significant throughput improvements (up to 1.6x) and dramatically faster strategy search times (9-21x) compared to state-of-the-art SPP systems like PipeDream and Piper.

Strengths

1. A Fundamental and Well-Motivated Contribution: The core idea of GPP is both elegant and, in retrospect, an obvious next step in the evolution of pipeline parallelism. The authors correctly identify a crucial mismatch between the increasingly complex, branched topologies of modern DNNs and the restrictive linear assumption of existing parallelism schemes. The paper's framing of the problem in Section 1 and the clear visual comparison in Figure 2 (page 3) make the motivation and the proposed solution exceptionally clear. This is not an incremental tweak but a fundamental shift in perspective from a 1D chain to a 2D graph.

2. Timeliness and High Relevance: The work is perfectly situated within current trends in machine learning. As the community moves towards "generalist" models (as cited with GPT-4, Chameleon, Gato in Section 1, page 2) that fuse information from different modalities, architectures with parallel processing streams are becoming the norm, not the exception. GPP provides a practical and performant solution to a problem of growing importance, making this work highly relevant to both the systems and machine learning communities.

3. Strong System Design and Evaluation: The paper goes beyond a theoretical proposal by developing and evaluating a complete system. The GRAPHPIPE system, with its series-parallel decomposition partitioner and micro-batch scheduler, demonstrates a thoughtful approach to solving the complex joint optimization problem. The evaluation is robust, using three distinct and relevant multi-branch models (MMT, DLRM, CANDLE-Uno) across a range of GPU counts. The results, particularly the consistent outperformance over strong baselines and the scaling behavior shown in Figure 6 (page 9), provide convincing evidence of GPP's practical benefits.

4. Connects Two Key Performance Levers: A particular strength is the clear demonstration of how GPP unlocks a virtuous cycle. By reducing pipeline depth, it not only cuts down on idle time but also reduces peak memory usage. This saved memory can then be reinvested into larger micro-batches, which improves the operational intensity and computational efficiency on modern accelerators. The case study in Section 7.5 (page 11) and the accompanying Figure 10 (page 12) beautifully illustrate this dual benefit.

Weaknesses

1. Limited Contextualization within Broader Parallel Computing: While the paper does an excellent job of positioning GPP against other DNN pipeline parallelism works (PipeDream, Piper), it misses an opportunity to connect with the broader history of task-graph scheduling in High-Performance Computing (HPC) and distributed systems. The problem of partitioning and scheduling a DAG of computations is a classic one, addressed by systems like Legion, Dask, and task-based runtimes using OpenMP/OmpSs. The paper would be strengthened by briefly acknowledging this lineage and more explicitly articulating the unique, domain-specific challenges of DNN training (e.g., the symmetric forward/backward pass structure, activation checkpointing/memory management) that necessitate a specialized solution like GRAPHPIPE rather than an off-the-shelf task scheduler.

2. Exploration of Topological Complexity: The models evaluated, while multi-branch, primarily exhibit a "fork-join" style of parallelism where independent branches run and then merge. The GPP framework appears general enough to handle more complex DAGs (e.g., a stage that feeds into two separate, non-converging downstream branches, or staggered dependencies). However, the evaluation does not explore these more intricate topologies. Discussing how the scheduler and partitioner would handle such cases would provide a more complete picture of GPP's generality and potential limitations.

3. Static Scheduling Assumption: The choice of a static micro-batch scheduler is practical and well-justified for the typical large-model training environment. However, this is a potential limitation. The paper could benefit from a brief discussion on the trade-offs of this decision and the potential for future work on dynamic or adaptive scheduling within the GPP framework, which might be useful in more heterogeneous hardware environments or for workloads with higher execution time variance.

Questions to Address In Rebuttal

1. Could the authors elaborate on the relationship between GPP and more general task-based parallel programming models like Legion or Dask? What are the key domain-specific challenges in DNN training (e.g., activation memory, gradient accumulation semantics) that prevent a direct application of these systems and necessitate the specific scheduler designed in GRAPHPIPE?

2. The evaluated models primarily feature a 'fork-join' topology. Could the authors comment on the applicability and potential performance of GPP on more complex DNN computation graphs, such as those with stages that have multiple, non-converging downstream paths or more intricate inter-stage dependencies?

3. The proposed micro-batch scheduler is static. Have the authors considered scenarios, perhaps involving conditional computation or heterogeneous clusters, where operator execution times might be variable? What are the trade-offs of the current static approach, and could the GPP framework accommodate more dynamic scheduling policies in the future?

Review Form: The Innovator (Novelty Specialist)

Summary

This paper introduces "Graph Pipeline Parallelism" (GPP), a novel scheme for pipeline-parallel training of Deep Neural Networks (DNNs). The central claim is that existing systems, which the authors categorize under "Sequential Pipeline Parallelism" (SPP), artificially linearize a DNN's computation graph, thereby missing opportunities for concurrent execution of independent branches. GPP, in contrast, preserves the inherent Directed Acyclic Graph (DAG) topology of the DNN when partitioning it into stages. This preservation allows for the concurrent execution of parallel stages, which the authors argue leads to a shorter effective pipeline depth, reduced activation memory, and consequently, higher throughput. The paper presents GRAPHPIPE, a system that embodies this GPP concept, featuring a novel topology-aware partitioner based on series-parallel decomposition and a scheduler that manages the resulting DAG of stages.

Strengths

The primary strength of this paper is the successful isolation and formalization of a new, intuitive, and seemingly overlooked parallelism strategy.

1. Novel Conceptual Abstraction: The core idea of GPP—partitioning a model into a DAG of stages rather than a linear sequence—is a clear and elegant conceptual contribution. While the notion of executing independent operations in parallel is fundamental, framing it specifically as a generalization of pipeline parallelism is novel. The distinction between SPP and GPP, as lucidly illustrated in Figure 2 (page 3), effectively carves out a new design point in the space of DNN parallelization strategies.

2. Efficient Instantiation of the Concept: The novelty extends beyond the GPP concept to its practical implementation. The pipeline stage partitioner described in Section 5, which uses series-parallel decomposition and dynamic programming, is a non-trivial and novel contribution. It provides a structured and efficient method for navigating the complex search space that GPP opens up. The fact that this targeted approach yields a 9-21x reduction in search time compared to more general planners (Table 1, page 10) demonstrates that the novelty lies not just in the what (GPP) but also in the how (the search algorithm).

3. Clear Differentiation from Canonical Prior Art: The authors correctly identify that the canonical pipeline parallelism systems like GPipe [11] and PipeDream [24, 25] are predicated on a linear stage abstraction. The proposed GPP is a genuine generalization of this model, and the paper's contribution is in being the first to explicitly define and build a system for this generalized case.

Weaknesses

From a novelty perspective, the primary weakness is the potential conceptual overlap with highly general, automated parallelism frameworks. The "delta" over this segment of prior art needs to be more sharply defined.

1. Overlap with General Auto-Parallelism Compilers: Systems like Alpa [48] or the earlier work on automatic dataflow graph partitioning [45] aim to automatically discover optimal parallelism strategies across inter- and intra-operator parallelism. It is conceivable that a sufficiently powerful general compiler could discover a GPP-like execution strategy as an emergent property of its optimization process. The paper acknowledges Alpa but does not sufficiently argue why formalizing GPP as a distinct strategy is superior to or fundamentally different from what these general frameworks could achieve. The novelty is presented as a new manual strategy, but its distinctness from the output of a fully automated one is not rigorously established.

2. Reliance on Series-Parallel Graph Assumption: The core partitioning algorithm (Algorithm 1, page 6) is built on series-parallel decomposition. The authors briefly mention that a non-series-parallel graph can be converted to an "arithmetically identical one" (Section 5, page 6), but this is a critical detail that is glossed over. The novelty and overhead of this conversion are unstated. If many modern architectures (e.g., those with complex skip connections that violate SP-graph properties) require this conversion, the practicality and performance of the novel partitioner might be impacted in ways not explored in the paper.

3. Nuanced Distinction from Flexible Planners: Piper [40] uses a "multidimensional planner" which can, in principle, create partitions that are not strictly sequential. For instance, it can group operators from different logical branches into a single stage. While GRAPHPIPE's explicit DAG execution is a clearer model, the degree to which Piper's search space already covers functionally similar partitions is unclear. The paper's novelty could be interpreted as a much more efficient heuristic for a search space that was, at least in part, previously explored by Piper, rather than an entirely new search space.

Questions to Address In Rebuttal

1. Could the authors please elaborate on the distinction between GPP and the potential output of a general auto-parallelism framework like Alpa [48]? Is the primary contribution a new target for such compilers to consider, or is it a strategy that is fundamentally outside their search capabilities? Clarifying this would help position the novelty of GPP more precisely in the context of automated parallelization.

2. The partitioner's reliance on series-parallel decomposition is a key design choice. Could the authors comment on the prevalence of non-series-parallel structures in modern, multi-branch DNNs? What is the specific conversion method used for such graphs, and what is its associated computational and performance overhead?

3. Regarding Piper [40], is it correct to say that Piper's planner could theoretically find a partition and schedule that is functionally equivalent to a GPP strategy, but is computationally intractable? Or is the GPP model of an explicit DAG of concurrently executing stages fundamentally outside of Piper's search space? A clearer articulation of the "delta" here would be valuable.

Paper Title: HALO: Loop-aware Bootstrapping Management for Fully Homomorphic Encryption Review Form: The Guardian

---

Summary

The authors introduce HALO, a compiler intended to automate and optimize bootstrapping for FHE programs containing loops, a notable limitation in prior work. The system first generates a "type-matched" loop by ensuring encryption status and levels of loop-carried variables are consistent across iterations, inserting bootstraps as needed for correctness. It then applies a series of optimizations: (1) packing multiple loop-carried variables into a single ciphertext to reduce the number of bootstraps, (2) partially unrolling short loops to better utilize the levels restored by bootstrapping, and (3) adjusting the target level of bootstrapping to match the consumption needs of the loop body, thereby reducing bootstrap latency. The authors evaluate HALO against their prior work, DaCapo, on seven machine learning benchmarks, claiming a 27% average execution speedup, alongside significant reductions in compilation time and code size.

Strengths

1. Problem Significance: The paper addresses a well-known and critical limitation in existing FHE compilers—the lack of native support for loops. Automating bootstrapping management within loops is a necessary step for making FHE practical for a broader class of algorithms.

2. Pragmatic Optimizations: The proposed optimizations (packing, unrolling, target level tuning) are sensible and directly target the primary overheads associated with naive bootstrapping in loops. The idea of reducing the bootstrap target level (Challenge B-3, page 5) is a particularly practical insight into the performance characteristics of the bootstrapping operation itself.

3. Compile-Time and Code-Size Improvements: The paper convincingly demonstrates that avoiding full loop unrolling leads to dramatic, order-of-magnitude reductions in compilation time and final code size (Table 6, page 10 and Table 7, page 11). This is a clear and undeniable benefit for developer productivity and resource management, especially for programs with a large number of iterations.

Weaknesses

1. The Baseline Comparison Is a Strawman: The central claim of a 27% performance speedup is predicated on a comparison with DaCapo, a compiler that is explicitly defined as lacking loop support and therefore must fully unroll the loop. This is not a fair or insightful comparison. Of course, a specialized loop-aware compiler has the potential to outperform a generic compiler forced to operate on a massive, unrolled block of straight-line code. The unrolled code presents an explosion of candidate points for bootstrapping, making optimization a much harder problem for the baseline system. The authors fail to justify why this is the appropriate state-of-the-art for this specific problem, rather than, for instance, a comparison against manually optimized loops which represent the practical alternative for an expert FHE programmer today.

2. Superficial Analysis of Performance Inconsistencies: The evaluation reveals that HALO is not universally superior. In the K-means benchmark (Figure 4e, page 10), DaCapo significantly outperforms HALO (by up to 12.4%). The paper dismisses this with a brief, unsubstantiated explanation: "DaCapo performs better than HALO because it optimizes the fully unrolled code... allowing the compiler to adapt the bootstrapping placement to the exact level of consumption." This is insufficient. A rigorous analysis would require a deep dive into the program structure of K-means, the specific bootstrapping decisions made by both compilers, and a clear, evidence-based explanation for why the unrolled approach is superior in this specific instance. Without this, the 27% average speedup figure appears to be a product of cherry-picked benchmarks that favor the authors' approach.

3. Insufficient Detail and Scrutiny of Optimization Heuristics: The description of the optimization strategies lacks depth. For example, "Level-aware Loop Unrolling" (Section 6.2, page 8) appears to be based on a simple heuristic comparing the multiplicative depth of the loop body (depth_used) to the maximum available depth (depth_limit). This seems overly simplistic and may not be robust. The paper does not discuss limitations, such as handling loops with internal control flow or more complex data dependencies that might complicate depth analysis. The novelty of these optimizations is also questionable, as packing and unrolling are well-known techniques; the contribution is their automation, but the sophistication of this automation is not clearly established.

4. Weak and Unsubstantiated Claims in the PCA Case Study: In the nested-loop case study (Section 7.4, page 11), the authors claim that for the (8,4) iteration count, DaCapo's performance degrades because its "filtering of the optimization space... filters out the better solution." This is a strong claim made without a shred of evidence. To substantiate this, the authors would need to show the set of candidate bootstrap points DaCapo considered, explain the filtering mechanism that led to a suboptimal choice, and demonstrate what the optimal choice was and why HALO's loop-based structure inherently finds it. As presented, it reads as a post-hoc justification for an observed result rather than a rigorous analysis.

5. Lack of Sensitivity Analysis: The entire evaluation is conducted using a single set of FHE parameters (Table 1, page 2). The performance trade-offs in FHE, particularly the cost of bootstrapping relative to other operations, are highly sensitive to the choice of parameters (e.g., security level, polynomial modulus degree, plaintext precision). The conclusions drawn in this paper may not hold under different parameter sets. A robust evaluation would demonstrate the impact of varying these parameters on the relative performance of HALO and the baseline.

Questions to Address In Rebuttal

1. Please justify the selection of "DaCapo on a fully unrolled loop" as the state-of-the-art baseline. Why is this a more valid comparison than comparing against a manually optimized FHE loop, which is what a practitioner would otherwise write?

2. Provide a detailed analysis of the K-means benchmark result (Figure 4e). What specific characteristics of this benchmark's computational graph allow the unrolled version to be optimized more effectively by DaCapo than HALO's loop-centric approach?

3. Regarding the PCA case study (Figure 5, Section 7.4), can you provide concrete evidence for the claim that DaCapo's heuristic "filters out the better solution"? Please detail the specific bootstrap insertion points considered and discarded by DaCapo that led to the observed performance degradation.

4. The loop unrolling strategy described in Section 6.2 appears to be based on a simple depth comparison. How does this heuristic handle loops with conditional branches or variable multiplicative depths per iteration? What are the precise limitations of this approach?

5. How would the performance advantage of HALO change if FHE parameters corresponding to a higher security level (e.g., 192-bit) were used, which would significantly alter the relative cost of bootstrapping? Please provide data or a well-reasoned argument on the sensitivity of your results to the core FHE parameters.

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper introduces HALO, a compiler for Fully Homomorphic Encryption (FHE) designed to address a critical, and largely unsolved, problem in the field: the efficient management of bootstrapping within programmatic loops. The authors correctly identify that prior state-of-the-art compilers, such as DaCapo, handle loops by fully unrolling them, leading to extreme inefficiencies in compilation time, code size, and an inability to support loops with dynamic iteration counts.

HALO's core contribution is a compilation strategy that treats loops as first-class citizens. It ensures correctness by matching the encryption status and noise levels of loop-carried variables across iterations. Furthermore, it introduces a suite of novel, loop-aware optimizations to mitigate the high cost of bootstrapping: (1) packing multiple loop-carried variables into a single ciphertext to reduce the number of bootstrap operations, (2) selectively unrolling short loops to better utilize the "recharged" levels post-bootstrapping, and (3) tuning the target level of bootstrapping to avoid unnecessary computational overhead. The empirical evaluation shows significant improvements over the unrolling-based approach, demonstrating a 27% average performance speedup in execution time, and, more dramatically, a geometric mean reduction of 209x in compilation time and 11x in code size.

Strengths

1. High Significance and Timeliness: The work addresses a fundamental bottleneck in the practical application of FHE. As the community moves from cryptographic primitives to building complex applications, the lack of robust support for control flow, particularly loops, has been a major barrier. HALO represents a conceptual leap from viewing FHE programs as static dataflow graphs (requiring full unrolling) to handling them as genuine, iterative programs. This is a crucial step towards making FHE programming more accessible and scalable.

2. Clear Problem Articulation: The authors do an excellent job in Section 3 ("Challenges," page 575) of distilling the core issues. The distinction between correctness challenges (type and level mismatches of loop-carried variables) and performance challenges (overhead from excessive or inefficient bootstrapping) is clear and compelling. The illustrative examples in Figures 2 and 3 effectively communicate the subtleties of the problem to a broader audience.

3. Elegant and Multi-faceted Solution: The proposed solution is not a single trick but a well-integrated set of techniques that address the identified challenges systematically.

   • The core idea of enforcing type and level consistency for loop-carried variables is a direct and necessary solution for correctness.
   • The optimization of packing loop-carried variables is particularly insightful. It correctly identifies that the number of bootstraps is a primary driver of overhead and provides a direct method to reduce it, transforming a per-variable cost into a single, shared cost.
   • The combination of level-aware unrolling and target-level tuning shows a sophisticated understanding of the performance trade-offs within the FHE execution model.

4. Strong Connection to the Broader Compiler Landscape: At its heart, this work is about applying classic compiler theory (data-flow analysis, loop-carried dependencies, loop optimizations) to the unique, resource-constrained domain of FHE. It successfully bridges these two fields, demonstrating how well-established compilation principles can be adapted to solve modern cryptographic challenges.

5. Impressive Empirical Results: The evaluation is thorough and the results are compelling. The comparison against DaCapo, the direct state-of-the-art, is the correct one. While the 27% execution speedup is significant, the orders-of-magnitude improvements in compilation time and code size are the most striking result. These metrics directly prove the unsustainability of the full-unrolling approach for any reasonably complex iterative program and validate HALO's methodology as the more viable path forward. The PCA nested-loop case study (Section 7.4, page 582) further strengthens the claim of generality.

Weaknesses

While the core contribution is strong, the paper could be strengthened by discussing the boundaries and limitations of the proposed approach.

1. Scope of Control Flow Support: The paper primarily focuses on for loops where the iteration count, even if large, is known or can be parameterized. The current design does not seem to accommodate more complex control flow, such as while loops that terminate based on a data-dependent condition or loops with early exits (break). These constructs are common in many iterative algorithms (e.g., convergence loops) and represent the next logical frontier for FHE compilers. Acknowledging this limitation and discussing the potential challenges (e.g., evaluating a condition on a ciphertext) would provide valuable context.

2. Generality of the Packing Optimization: The variable packing technique is a key strength, but its applicability may be constrained. The paper does not discuss the scenario where the number of loop-carried variables exceeds the packing capacity of a single ciphertext (determined by the polynomial degree N). A discussion on how HALO would handle such cases (e.g., creating multiple packed ciphertexts) and the performance implications thereof would make the analysis more complete.

3. Details of the Optimization Heuristics: The paper describes what optimizations are performed (e.g., unrolling, target level tuning) but provides limited insight into how the compiler decides when and how to apply them. For instance, how is the optimal unroll factor determined? Is there a cost model that weighs the benefit of utilizing more levels against the overhead of a larger loop body? While a full exploration is likely beyond the scope of one paper, a brief discussion of the decision-making logic would be beneficial.

Questions to Address In Rebuttal

1. Could the authors clarify the current limitations of HALO's loop analysis? Specifically, can it handle while loops or loops with conditional break statements? If not, what are the primary challenges to extending the framework to support such control flow structures?

2. Regarding the loop-carried variable packing optimization, how does HALO scale when the number of variables is too large to fit into a single ciphertext? Does it partition them into multiple packed ciphertexts, and what is the expected performance impact?

3. Could you elaborate on the decision-making process within the HALO compiler for applying the level-aware unrolling and target level tuning optimizations? Is this guided by a performance model, or is it based on a set of pre-defined heuristics?

Reviewer: The Innovator (Novelty Specialist)

Summary

This paper introduces HALO, a compiler for Fully Homomorphic Encryption (FHE) that provides, for the first time, automated bootstrapping management for programs containing loops without resorting to full loop unrolling. The central claim of novelty is this loop-aware compilation strategy. To achieve this, the authors propose several techniques: (1) a type-matching system to ensure correctness by unifying the encryption status and level of loop-carried variables across iterations; (2) a set of optimizations to reduce the high cost of bootstrapping within loops, namely packing multiple loop-carried variables into a single ciphertext to minimize the number of bootstrap operations, level-aware unrolling to better utilize the "level budget" granted by a bootstrap, and tuning the target level of the bootstrap operation to avoid unnecessary computational overhead. The authors evaluate HALO against a state-of-the-art FHE compiler (DaCapo) that relies on full unrolling, demonstrating significant improvements in compilation time and code size, as well as a notable performance speedup.

Strengths

The primary strength of this paper lies in its novel approach to a well-defined and critical problem in FHE usability.

1. Addressing a Fundamental Limitation: The prior art in FHE compilers, such as DaCapo [13] and HECO [59], is largely limited to programs with simple control flow or loops that can be fully unrolled. This severely restricts the complexity of programs that can be practically developed and compiled. HALO's core proposal—to generate a compact, single loop body that is valid for an arbitrary number of iterations—is a genuinely new and significant contribution to the field of FHE compilers. It moves the state-of-the-art from handling straight-line code (or what can be converted to it) to handling general iterative structures.

2. Novel Application of Existing Primitives for Optimization: While the constituent optimization ideas are inspired by well-known concepts, their application to the unique resource management problem of FHE loops is novel and insightful.

   • Packing Loop-Carried Variables (Section 4.2): Standard FHE schemes have long used packing to enable SIMD operations on data. However, applying this technique to aggregate multiple distinct loop-carried variables into a single ciphertext with the specific goal of reducing N bootstraps to one per iteration is a clever and previously unexplored optimization strategy in this context.
   • Level-Aware Unrolling (Section 6.2): Loop unrolling is a classic compiler optimization. HALO re-purposes it in a novel way: not for instruction-level parallelism, but as a tool to amortize the high fixed cost of bootstrapping. The heuristic of unrolling just enough to fully utilize the levels restored by a bootstrap is a new, domain-specific application of the technique tailored to FHE's unique execution model.
   • Bootstrap Target Level Tuning (Section 6.3): The observation that bootstrapping to a lower level is faster (Table 3) is known, but incorporating this as an automated compiler pass that analyzes the level consumption within the loop body to select an optimal, non-maximal target level appears to be a new contribution compared to the baseline established by DaCapo [13].

Weaknesses

From a novelty perspective, the weaknesses are less about flaws and more about the nature of the contribution.

1. Engineering Novelty over Foundational Novelty: The paper's contribution is primarily one of systems engineering and clever synthesis. The underlying primitives—the CKKS FHE scheme, bootstrapping, packing, and loop unrolling—are all pre-existing. The novelty lies in being the first to assemble and adapt these pieces to solve the specific problem of FHE loops. This is a valuable and non-trivial contribution, but it is not a new fundamental algorithm or cryptographic primitive.

2. Adaptation of Standard Compiler Theory: The correctness requirement of matching the "type" of loop-carried variables between iterations (Section 3.1) is an application of standard data-flow analysis principles found in any modern compiler textbook. The challenge and novelty here are not in the abstract concept but in defining the FHE-specific "type system" (i.e., encryption status and level) and building the analysis to handle it, which the paper correctly identifies.

Questions to Address In Rebuttal

1. The optimization of packing loop-carried variables to reduce bootstrap count is compelling. To solidify the claim of novelty, could the authors confirm if any prior work, perhaps in manual FHE program optimization or in different FHE compiler frameworks, has proposed or implemented a similar strategy for managing loop-carried state?

2. The paper contrasts its dynamic bootstrap target level tuning against DaCapo [13], which is claimed to use a fixed maximum level. While this establishes a delta against the most direct competitor, is this concept entirely new to the FHE space? Has the idea of selecting a lower bootstrap level to save latency been discussed in the context of other FHE libraries or optimization frameworks, even if not automated within a compiler?

3. The level-aware unrolling heuristic appears effective. Was this the first heuristic considered? The paper presents it as a solution to "wasted levels" (Challenge B-2, page 5). Did the authors explore alternative approaches to managing this waste, and if so, why was level-aware unrolling chosen as the most promising path? This would help clarify the design space and the novelty of the chosen solution.

Paper Title: Helix: Serving Large Language Models over Heterogeneous GPUs and Network via Max-Flow Reviewer Persona: The Guardian (Adversarial Skeptic)

---

Summary

The paper presents Helix, a system for serving Large Language Models (LLMs) on heterogeneous GPU clusters, potentially spanning multiple geographic regions. The central thesis is that the complex, entangled problems of model placement and request scheduling can be jointly optimized. To achieve this, the authors formulate the serving problem as a max-flow problem on a directed graph representing the cluster's compute and network resources. This formulation is then solved using a Mixed Integer Linear Programming (MILP) algorithm to find a purportedly optimal model placement. At runtime, a weighted round-robin scheduler assigns requests to per-request pipelines derived from the max-flow solution. The authors claim significant improvements in throughput (up to 3.3x) and reductions in latency compared to baseline approaches, evaluated on both a prototype and a simulator.

Strengths

1. Formalization of the Problem: The formalization of the heterogeneous LLM serving problem as a max-flow optimization is a notable contribution. Abstracting the cluster's heterogeneous components (GPUs, network links) into a unified graph with capacities is an elegant way to model the system's constraints.

2. Joint Optimization Goal: The attempt to jointly optimize model placement and request scheduling is ambitious and addresses a genuine challenge in distributed serving systems. Separately optimizing these two highly dependent tasks, as is common, can lead to the sub-optimal outcomes illustrated in the paper's motivating example (Figure 1, Page 4).

3. Comprehensive Evaluation Scope: The authors evaluate their system across a variety of simulated and real hardware configurations, including single-cluster, geo-distributed, and high-heterogeneity scenarios. The use of both online and offline serving metrics provides a broader view of system performance.

Weaknesses

My primary concerns with this work center on the practicality of the proposed MILP-based approach, the oversimplification of critical system components in the model, and the rigor of the evaluation baselines.

1. Feasibility and Scalability of the MILP Solver: The paper's core claim of finding an "optimal" solution hinges on the MILP solver. However, MILP is NP-hard, a fact the authors conveniently downplay. The entire approach is made tractable only through a series of "optimizations" described in Section 4.5 (Page 7) that fundamentally undermine the claim of optimality for any non-trivial cluster.

   • Pruning and Heuristic Hinting: The necessity of pruning network connections and using heuristic solutions as starting points suggests the search space is intractable. This transforms the "optimal" planner into a computationally expensive, guided heuristic search. The claim of optimality is therefore void for the systems under test.
   • Unreported Solve Times and Optimality Gaps: The most critical omission is the lack of data on the MILP solver's performance for the main experimental setups. Section 6.9 and Figure 12 (Page 13) show that for a trivial 10-node cluster, confirming optimality takes over an hour. What were the solve times and, more importantly, the optimality gaps for the 24-node and 42-node clusters evaluated in Sections 6.3, 6.4, and 6.5? Without this data, the quality of the "optimized" solution is unknown and the central claim of the paper is unsubstantiated.

2. Oversimplification in System Modeling: The max-flow abstraction, while elegant, makes simplifying assumptions that may not hold in production environments.

   • Naive KV-Cache Management: The KV-cache estimation described in Section 5.2 (Page 8) is critically flawed. The system relies on an "estimation of KV-cache usage... using average output length." Real-world request distributions have high variance in output lengths. A single long-running request can exhaust a GPU's memory, causing a cascade of failures or offloading that is not captured by this simplistic model. This is a significant practical weakness that compromises the system's robustness.
   • Static Throughput Profiling: The model relies on a one-time, static profiling of GPU and network throughput (Section 4.3, Page 5). Real systems exhibit dynamic behavior due to thermal throttling, background processes, network jitter, and contention. The static graph capacities cannot account for this, making the "optimal" plan potentially fragile.

3. Weakness of Baselines and Evaluation Rigor: The impressive performance gains reported for Helix may be inflated due to the choice and implementation of baselines.

   • Re-implementation of Baselines: The authors have re-implemented the baseline systems (Swarm, SP). It is unclear if these re-implementations capture the full behavior and optimizations of the original systems or if they represent weaker versions that are easier to outperform. The case study in Figure 9b (Page 11), for instance, shows Swarm creating an obvious bottleneck. Is this an inherent flaw of Swarm's algorithm or an artifact of the authors' specific implementation and tuning?
   • Heavy Reliance on Simulation: Two of the three main evaluation scenarios (geo-distributed and high-heterogeneity) are conducted exclusively in a simulator. While the authors claim a <5% error rate (Page 9), this validation appears limited to aggregate metrics on a single cluster configuration. The simulator's ability to accurately model complex network dynamics like congestion collapse in geo-distributed settings is not proven, casting doubt on the results in Section 6.4.

Questions to Address In Rebuttal

1. For the 24-node and 42-node cluster experiments presented in the paper, please provide the exact MILP solver runtimes and the final optimality gap reported by the solver upon termination. How close are these "optimized" solutions to the theoretical optimum?

2. Regarding the KV-cache estimation in Section 5.2: What is the mechanism for handling requests whose actual output length significantly deviates from the average? Have the authors measured the rate of KV-cache misses or offloading under real-world trace variance, and what is its performance impact? A system that relies on averages is not robust.

3. Can the authors provide more details on the validation of the simulator beyond the aggregate error rate on a single cluster? Specifically, how was the simulator validated for complex cross-contention scenarios and network dynamics prevalent in geo-distributed settings?

4. The request scheduling in Section 5.1 is based on the flow values from the static max-flow solution. How does this static scheduling plan adapt to dynamic shifts in the workload (e.g., a sudden increase in requests with long prompts) that might favor different paths through the cluster?

Reviewer: The Synthesizer (Contextual Analyst)

---

Summary

This paper introduces Helix, a system designed for serving Large Language Models (LLMs) on heterogeneous and geographically distributed GPU clusters. The core contribution is the novel formulation of this complex resource allocation problem as a maximum flow problem on a directed graph. The authors abstract the entire cluster—comprising GPUs with varying compute/memory capacities and network links with different bandwidths—into a flow network. They then employ a Mixed Integer Linear Programming (MILP) solver to find a provably optimal model layer placement that maximizes this flow, which directly corresponds to the cluster's maximum theoretical throughput.

At runtime, Helix operationalizes this offline plan through a flexible "per-request pipeline" model, where a flow-guided scheduler (Interleaved Weighted Round-Robin) probabilistically routes requests along the high-capacity paths identified by the max-flow solution. The paper presents a comprehensive evaluation showing that this principled, optimization-based approach significantly outperforms heuristic-based baselines in terms of throughput and latency.

---

Strengths

1. Elegant and Principled Formulation: The standout strength of this paper is its core idea: abstracting the entire heterogeneous serving problem into a classic max-flow formulation. This is an elegant leap that moves the field away from ad-hoc heuristics towards a more formal, provably optimal framework. By modeling GPUs, compute capacity, and network bandwidth as nodes and edge capacities, the authors unify disparate system components into a single, analyzable model. This is a significant intellectual contribution.

2. Holistic, Joint Optimization: The paper correctly identifies that model placement and request scheduling are "highly entangled tasks" (Abstract, page 1). The Helix framework provides a holistic solution by using the MILP to solve the placement problem globally and offline, and then using the resulting flow values to directly inform the online scheduling decisions. This creates a powerful synergy between the long-term strategic placement and the short-term tactical scheduling, a connection often missing in prior work.

3. High Problem Relevance and Timeliness: The work addresses a critical, real-world challenge. As LLMs grow, deploying them requires aggregating vast amounts of compute, and the reality of cloud data centers (as shown in Table 2, page 1) and GPU supply chains is heterogeneity. Systems that can efficiently harness a motley crew of hardware are not just an academic curiosity; they are an economic necessity. Helix provides a concrete answer to this pressing problem.

4. Strong, Insightful Evaluation: The experimental results are not only strong but also well-analyzed. The authors go beyond simply presenting performance graphs. The case studies in Section 6.6 (Model Placement Deep Dive, page 11) and Section 6.7 (Request Scheduling Deep Dive, page 12) are particularly effective. Figure 9b, for instance, provides a clear visual intuition for why Helix's placement avoids the bottlenecks that plague heuristic methods like Swarm. Similarly, the analysis in Figure 12 (page 13) of the MILP solver's progress over time is excellent, showing that near-optimal solutions are found quickly, bolstering the practical viability of the approach.

---

Weaknesses

While the core idea is strong, its practical application brings to light some limitations that are characteristic of optimization-based systems.

1. Static, Offline Optimization: The MILP-based model placement is performed once, offline, for a given cluster configuration. The real world is dynamic; nodes can fail, be preempted, or be added to the cluster. Workload characteristics can also shift over time. The current framework does not seem to have a low-cost mechanism to adapt to such changes short of a full, and potentially slow, re-solve of the MILP. The paper would be strengthened by a discussion of how the system might handle such dynamism.

2. Scalability of the MILP Solver: The authors acknowledge that the MILP can be slow and propose several practical optimizations (Section 4.5, page 7). While effective for the tested clusters (up to 42 nodes), the computational complexity of MILP will eventually become a bottleneck for very large-scale systems (hundreds or thousands of nodes). The suggestion to "partition the nodes into multiple smaller clusters" is a pragmatic heuristic, but it moves away from the global optimality that is the hallmark of the approach. The paper's primary contribution is strongest in the medium-scale regime where global optimization is still tractable.

---

Questions to Address In Rebuttal

1. Adaptivity to Dynamic Conditions: Could you elaborate on how Helix would handle dynamic changes in the cluster, such as a sudden node failure or the addition of new resources? Would a partial or incremental re-optimization be possible, or would the system need to fall back to a sub-optimal state while a full new plan is computed?

2. Sensitivity to Profiling Accuracy: The entire optimization framework relies on profiled throughput numbers for GPU compute and network links. How sensitive is the quality of the final MILP-derived placement to inaccuracies or noise in these initial measurements? For instance, if the actual performance of a GPU under load deviates by 10% from its profiled value, how much does that degrade the "optimality" of the system's performance?

3. Cost of Complexity vs. Heuristics: The MILP framework is significantly more complex to implement and solve than the heuristic baselines. Your evaluation shows it provides substantial benefits over simple heuristics (like those in Swarm). However, could a more sophisticated, but still heuristic-based, graph algorithm (e.g., a greedy algorithm that is aware of both node capacity and cut-size) achieve, say, 95% of the performance of the optimal MILP solution at a fraction of the computational cost? What is your perspective on the trade-offs in this design space?

Paper: Helix: Serving Large Language Models over Heterogeneous GPUs and Network via Max-Flow Reviewer Persona: Innovator (Novelty Specialist)

---

Summary

The paper presents Helix, a system designed to serve Large Language Models (LLMs) on clusters composed of heterogeneous and potentially geo-distributed GPUs. The central claim to novelty is the formulation of this complex resource allocation challenge as a max-flow problem on a directed, weighted graph. In this graph, nodes represent GPU compute capacity, and edge capacities represent network bandwidth. The authors then employ a Mixed Integer Linear Program (MILP) to find the optimal model layer placement strategy that maximizes the total flow through this graph, which corresponds to maximizing the cluster's token throughput. The runtime component uses the solution from this optimization to schedule requests along "per-request pipelines" derived from the flow paths.

---

Strengths

The primary strength of this paper is its novel conceptualization of the heterogeneous LLM serving problem. My analysis of prior art confirms that while the constituent components—max-flow algorithms and MILP solvers—are classical operations research tools, their synthesis and specific application to model token throughput in this context is new.

1. Novel Formulation: The key contribution is the abstraction of the entire serving system (compute nodes, network links, model layers) into a flow network. This provides a principled, mathematical framework for a problem that has largely been addressed with heuristics. It elegantly unifies the highly-entangled tasks of model placement (which defines the graph's structure and capacities) and request scheduling (which defines the flow paths) into a single, cohesive optimization framework. This is a significant conceptual advance over prior work.

2. Clear Delta from Prior Art: The authors correctly position their work against existing systems. Unlike systems for homogeneous clusters (e.g., vLLM, Orca), Helix tackles heterogeneity head-on. Compared to decentralized approaches like Petals [5], Helix leverages global knowledge of a managed cluster to achieve a globally optimal solution, a distinct and more powerful model for non-volunteer infrastructure. Crucially, its distinction from training-focused systems like SWARM [45] or heuristic-based serving systems (such as the concurrent work HexGen [19]) is the pursuit of a provably optimal placement via a formal mathematical model, rather than relying on greedy or local decision-making.

---

Weaknesses

My critique is centered on the practical implications of the chosen novel approach and whether the abstraction is fully comprehensive.

1. High Complexity for Novelty: The reliance on an MILP solver is the most significant weakness from a novelty-justification standpoint. MILP is NP-hard, a fact the authors acknowledge in Section 4.5 (page 7). While this complexity is amortized by presenting it as a one-time, offline planning step, the scalability of this approach is a genuine concern. The novelty comes at a steep computational price. The evaluation is performed on clusters up to 42 nodes; it is unclear if this novel method remains tractable for the hundreds or thousands of nodes that constitute large-scale production clusters. The proposed pruning techniques are heuristics applied to an otherwise optimal method, which slightly dilutes the purity of the "optimal" claim in practice.

2. Fidelity of the Abstraction: The abstraction to a max-flow problem, while elegant, may oversimplify certain system dynamics that are critical in real-world serving. The model relies on static, pre-profiled values for GPU compute throughput and network bandwidth (Section 4.3, page 5). This fails to capture dynamic effects such as network congestion from cross-traffic, transient slowdowns in GPU performance due to thermal throttling, or the complex interplay of KV cache memory management with compute performance. The novelty of the formulation is powerful, but its robustness to real-world system "noise" not captured in the model is questionable.

3. Redefinition of Existing Concepts: The concept of "per-request pipelines" (Section 5.1, page 8) is presented as a contribution. However, this appears to be a natural and direct consequence of routing individual units of work (requests) through a flow network. Any system that performs path-based routing in a network could be described in this manner. The true novelty lies in the generation of the underlying graph and its capacities via MILP, not necessarily in the scheduling paradigm itself, which seems to be a standard application of weighted round-robin over valid paths in the solution graph.

---

Questions to Address In Rebuttal

1. Scalability of the Core Method: Given that MILP solving time is known to be exponential in the worst case, how does the proposed optimization approach scale to clusters of hundreds of nodes? At what point does the "offline" planning time become prohibitive (e.g., days or weeks), rendering the method impractical for dynamic cloud environments where cluster configurations can change frequently?

2. Justification of Complexity vs. Heuristics: The paper mentions concurrent work HexGen [19] (Section 7, page 14), which uses heuristics for a similar problem. Could the authors provide a more direct comparison or theoretical argument on the trade-offs? Specifically, how much of the reported 3.3x performance improvement is attributable to the optimality of the MILP solution versus what could be achieved by a state-of-the-art, but computationally cheap, heuristic? Is the substantial computational overhead of MILP always justified?

3. Robustness of the Static Model: The optimization relies on a one-time profiling of the cluster. How sensitive is the performance of the resulting "optimal" placement to variations in network bandwidth or transient GPU performance degradation? For example, if a key inter-region link's bandwidth drops by 30% from its profiled value, does the system suffer catastrophic congestion, or does it degrade gracefully? This speaks directly to whether the novel static formulation is viable for dynamic real-world systems.

Paper: H-HOUDINI: Scalable Invariant Learning Reviewer: The Guardian (Adversarial Skeptic)

---

Summary

The authors present H-HOUDINI, a framework intended to scale inductive invariant learning by replacing monolithic SMT queries with a hierarchical, incremental approach based on relative inductivity. The core idea is to decompose the proof of a target property into a series of smaller proof obligations that correspond to the dataflow dependencies within the hardware design. The framework is instantiated as VELOCT to solve the Safe Instruction Set Synthesis Problem (SISP) for hardware security. The authors evaluate VELOCT on the RISC-V Rocketchip and, notably, on several variants of the out-of-order BOOM core, claiming significant performance improvements over prior monolithic approaches and the first successful automated invariant learning for a design of BOOM's complexity.

Strengths

1. The fundamental concept of decomposing a monolithic invariant proof into a hierarchy of smaller, relatively inductive proofs that mirror the circuit's cone-of-influence is a sound and compelling strategy for tackling the scalability bottleneck of SMT-based verification.
2. The successful application of the framework to the BOOM processor, a notoriously complex open-source out-of-order design, is a significant empirical result. Prior work has struggled to scale to this level, so demonstrating any automated analysis is a notable achievement.
3. The framework's design inherently supports a high degree of parallelism, which is a practical and necessary feature for analyzing large hardware designs.

Weaknesses

My primary concerns with this work center on the significant overstatement of its automation, the fragility of its formal arguments, and the heuristic nature of its core components, which call into question the generality and robustness of the approach.

1. The Claim of "Mostly Push-Button" Automation is Substantially Undermined by the Requirement for Expert Annotation. The paper claims the tool is "(mostly) push-button" (Abstract) and learns an invariant for Rocketchip with "no expert annotations" (Abstract). However, the success on the more complex and significant BOOM target is contingent on "modest expert annotations" (Section 6.2). This "modest" effort involves a graduate student manually identifying and annotating valid bits in multiple microarchitectural structures (ALU/CSR/FPU issue buffers, etc.) for "example masking" (Section 5.2.1) and manually adding predicates for micro-ops (InSafeUop) and ALU opcodes (Section 6.2). This is a critical, non-trivial manual intervention. The anecdotal measure of "less than a day" is not a rigorous metric of effort; for a different, more complex, or less well-documented design, this could be a multi-week reverse-engineering task requiring deep microarchitectural expertise. This manual effort effectively encodes a partial invariant into the system, guiding the "learning" process in a way that significantly weakens the central claim of automated invariant discovery.

2. The Soundness of the Entire Framework Relies on the Quality of Positive Examples. The soundness of the hierarchical decomposition, as presented in Section 3.1, hinges on the "Premise for Soundness (P-S)": each partial invariant H_i must be consistent with all positive examples. This premise is used to ensure the stepwise-proven H_i are not contradictory. However, this only guarantees non-contradiction with respect to the sampled examples, not the entire space of valid behaviors. If the positive examples are not sufficiently diverse, the algorithm could derive a set of locally consistent but globally contradictory partial invariants, leading to an unsound final invariant H that is vacuously true. The paper provides no formal argument or empirical evidence to suggest the example generation strategy is sufficient to prevent this.

3. The Predicate Mining and Abduction Oracles are Heuristic and Potentially Incomplete.

   • The predicate mining oracle O_mine (Algorithm 2, Section 5.1.2) generates candidate predicates based on properties that hold true across all provided positive examples (e.g., a register always being equal across two runs, or holding a constant value). This is a fragile heuristic. A single, valid but idiosyncratic positive example could prevent a necessary predicate from being mined, causing the entire proof to fail. The paper does not provide a formal argument for the completeness of this oracle—i.e., that it is guaranteed to generate the necessary predicates if an invariant exists within the language.
   • The abduction oracle O_abduct (Section 3.2.3) relies on extracting a minimal UNSAT core. The authors note that cvc5 "guarantees locally minimal unsat cores." This is a key weakness. The choice of UNSAT core can dramatically influence the subsequent search, and a "locally minimal" core is not guaranteed to be the simplest or most general explanation (abduct). The performance and even success of the algorithm may be highly sensitive to the SMT solver's non-deterministic core extraction heuristics, a dependency which is not explored.

4. The Evaluation is Incomplete, Weakening the Central Claims. In Section 6.4, the authors state they were "unable to verify the safety of the auipc instruction" on BOOM due to unexpected variable timing behavior. They subsequently "leave investigating why to future work." This is a critical admission. The work cannot claim to have solved the SISP for BOOM if it fails on a standard, non-privileged ISA instruction. This failure suggests a potential blind spot in the predicate language or the learning methodology when faced with certain microarchitectural timing patterns, which is a more fundamental issue than an incomplete run.

Questions to Address In Rebuttal

1. Please provide a more rigorous, quantitative characterization of the "expert annotation" effort required for BOOM. How many lines of annotation code were required? How many distinct hardware signals needed to be identified? How can the authors justify the "mostly automated" claim when this manual, expert-driven effort appears critical to success on the paper's main target?

2. The soundness of the approach (Section 3.1) relies on positive examples to ensure partial invariants are not contradictory. Please provide a more robust argument for why this is sufficient. How can you provide confidence that the generated examples are comprehensive enough to prevent the learning of a vacuous invariant that holds for the examples but not for all possible safe executions?

3. The failure to verify auipc on BOOM (Section 6.4) is a significant gap. Does this failure indicate a fundamental limitation in your predicate language or learning algorithm? Why should the committee be confident that this approach is generalizable if it fails on a standard instruction in its primary case study?

4. How sensitive is the H-HOUDINI algorithm, in terms of both performance and its ability to find an invariant, to the SMT solver's UNSAT core generation strategy? Given the reliance on "locally minimal" cores, what is the risk that the algorithm explores a suboptimal proof path or fails entirely due to the solver's heuristic choices?

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper introduces H-HOUDINI, a novel and compelling algorithm for scalable inductive invariant learning. The core contribution is a powerful synthesis of two major paradigms in formal verification: the predicate abstraction and example-driven nature of Machine Learning Inspired Synthesis (MLIS) and the incremental, structure-aware nature of SAT-based methods like IC3/PDR. H-HOUDINI cleverly decomposes the expensive, monolithic SMT queries typical of MLIS into a hierarchy of smaller, localized, and highly parallelizable relative inductive checks. This decomposition is guided by the program or circuit's dataflow structure (specifically, the 1-step cone of influence), effectively creating a "white-box" MLIS framework.

The authors instantiate this algorithm in a tool called VELOCT to tackle the challenging Safe Instruction Set Synthesis Problem (SISP) in hardware security. The empirical results are exceptional: VELOCT learns an invariant for the RISC-V Rocketchip core over 2800x faster than the state-of-the-art and, most impressively, is the first tool to successfully learn invariants for the entire family of complex RISC-V BOOM out-of-order cores, a task previously considered intractable for this class of automated tools.

Strengths

1. A Powerful Conceptual Bridge: The most significant strength of this work is its elegant conceptual contribution. It identifies a fundamental tension between the scalability of SAT-based "white-box" incremental methods and the search-space pruning power of MLIS's "black-box" approach. The proposed solution—using the design's structure to guide a divide-and-conquer learning process—is a truly insightful hybrid. It feels less like a heuristic and more like a principled way to combine the best of both worlds. The connection made in the related work (Section 7, page 11) to RICE-learning (Relative ICE) correctly positions this as a foundational step forward for MLIS-based techniques.

2. Landmark Scalability Result: The paper doesn't just present a new idea; it demonstrates its power on a problem of recognized difficulty. Scaling automated invariant synthesis to an out-of-order processor like BOOM is a major achievement for the hardware verification community. Prior work often stops at simpler, in-order cores. By successfully analyzing a design with over 133K state bits (MegaBOOM), the authors have significantly pushed the boundary of what is considered possible, transforming the problem from "impossible" to "achievable in hours." This result alone makes the paper highly significant.

3. Excellent Problem Choice and Grounding: Applying H-HOUDINI to the SISP for constant-time verification is a perfect choice. It's not a toy problem but a critical challenge in hardware security, with direct implications for building trustworthy systems. This grounding in a practical, high-impact domain makes the work immediately relevant and compelling.

4. Clear and Effective Exposition: The core idea of hierarchical decomposition is explained very clearly, both conceptually with the AND gate example (page 2) and formally with Algorithm 1 (page 5). The worked example in Appendix C (pages 14-16) is particularly helpful for building intuition about how the recursive abduction, backtracking, and memoization fit together.

Weaknesses

While the core idea and results are outstanding, there are areas where the framework's current limitations and dependencies could be further explored. My perspective here is not to diminish the contribution but to contextualize its current instantiation and highlight avenues for future work.

1. Reliance on Manual Annotation and Oracles: The paper claims H-HOUDINI is a "(mostly) push-button" algorithm, but the successful application to BOOM required a "modest number of annotations" (Section 6.2, page 9). These include augmenting the predicate language (e.g., InSafeUop) and, critically, annotating valid bits for "example masking." While this is a practical and necessary step to achieve the result, it represents a knowledge-transfer bottleneck. The power of the general H-HOUDINI framework is somewhat tied to the quality of the domain-specific Omine oracle and these annotations.

2. Demonstrated Generality: The work is masterfully demonstrated on a 2-safety (non-interference) property. This relational structure is a natural fit for the product-construction methodology used. It is less clear how the H-HOUDINI framework would apply to other important verification tasks, such as verifying functional correctness properties or liveness properties. The core hierarchical idea seems general, but its instantiation would require fundamentally different predicate languages and oracle implementations, the challenges of which are not fully explored.

3. Sensitivity to Positive Examples: The performance and correctness hinge on the quality of the positive examples used to prune the predicate space and avoid backtracking. The paper notes that backtracking occurs due to non-exhaustive examples (Section 6.3, page 10, Figure 5). While the strategy of simulating instructions seems effective, the framework's robustness in the face of a poor or incomplete set of examples is an important practical question. A few "bad" examples could potentially prune necessary predicates, leading to failure.

Questions to Address In Rebuttal

1. The Path to Full Automation: The expert annotations required for BOOM were key to success. Could the authors comment on the prospects for automating this "expert-in-the-loop" step? For instance, could static analysis of the RTL or a preliminary data-flow analysis automatically identify structures like instruction queues and their corresponding "valid" bits to guide example masking? How much of this domain knowledge could be formalized and integrated into the Omine oracle?

2. Beyond 2-Safety: The paper's conclusion rightly points to exploring other properties as future work. Could the authors elaborate on the conceptual challenges of applying H-HOUDINI to a standard functional correctness property (e.g., proving that a pipelined processor correctly implements the ISA specification)? What would the Ptarget, the predicate language, and the "positive examples" look like in that context?

3. Robustness of Example-Driven Learning: The backtracking analysis in Figure 5 is insightful. Could you provide more intuition on the failure modes? When H-HOUDINI fails to find an invariant (or backtracks excessively), is it typically due to a missing predicate in the language, insufficient positive examples missing a corner case, or fundamental scalability limits of the SMT solver on certain sub-problems? Understanding this could help guide future users.

Innovator Review Form

Isolate the Novel Claim: The authors propose H-HOUDINI, an algorithm for scalable inductive invariant learning. The central novel claim is the synthesis of two distinct verification paradigms: the predicate-abstraction and example-guided search from Machine Learning Inspired Synthesis (MLIS), and the incremental, relatively inductive checks from white-box, SAT-based methods like IC3/PDR.

The precise mechanism claimed as novel is the replacement of the traditional monolithic inductivity check (H => H') with a hierarchical decomposition. This process involves: 1. Identifying a target predicate (P_target). 2. Using the 1-step cone-of-influence (COI) to find relevant predicates from a universe (P_V). 3. Synthesizing a "local" abduct A using a relative inductive query (A ^ P_target => P_target'). 4. Recursively and parallelly solving for the inductivity of each predicate within the abduct A.

This creates a property-directed, recursive "wavefront" of smaller, parallelizable SMT checks that compose to prove the inductivity of the global invariant by construction, avoiding any single monolithic check.

Execute a "Prior Art" Search: The foundational concepts leveraged in this work are well-established.

1. MLIS: The core MLIS algorithm, HOUDINI [28], is over two decades old. Its use of a predicate universe and filtering based on counterexamples is the baseline.
2. Incremental/Relative Inductive Learning: The idea of using relative induction to incrementally build invariants is the core of IC3/PDR [8, 22]. These methods, however, are typically driven by generalizing negative counterexamples, not by a predefined predicate abstraction guided by positive examples.
3. Abductive Inference for Invariants: The use of abduction and interpolants to infer strengthening predicates is also not new, with notable prior work by McMillan [39] on interpolants and Dillig et al. [18] on abductive inference for loop invariants.
4. Relative ICE (RICE) Learning: The most proximate prior art is the concept of RICE-learning proposed by Vizel et al. in "IC3-flipping the E in ICE" (VMCAI 2017) [48]. They explicitly proposed replacing the monolithic inductive query in the ICE-learning framework (a generalization of MLIS) with a relative inductive query.

Evaluate the Delta: The core idea of "blending" MLIS with relative induction is therefore not fundamentally new; it was conceptualized as RICE-learning. The authors rightly acknowledge this in their Related Work section (Section 7, page 11).

However, the novelty of H-HOUDINI is not the concept, but the algorithm. Vizel et al. introduced the conceptual framework of a "RICE-learner" but did not provide a concrete, scalable, hierarchical algorithm for implementing it. H-HOUDINI is, to my knowledge, the first work to present such an algorithm. The "delta" over prior art is the specific, structured methodology for decomposing the problem:

• COI-Guided Decomposition: The explicit use of the 1-step COI to structure the recursive search is a novel and practical mechanism for applying the RICE concept to hardware verification.
• Hierarchical Abduction: While abduction for invariants exists, the structured, DFS-like recursive application of abduction to build up the invariant piece by piece is a new algorithmic pattern in this context.
• Scalability via Parallelism and Memoization: The algorithm is designed from the ground up for parallelism and memoization within its recursive structure. This is a significant engineering and algorithmic contribution that moves the RICE concept from theory to a highly practical tool.

In summary, H-HOUDINI does not invent a new category of learning but provides the first practical and scalable algorithmic blueprint for an existing, high-level idea.

Analyze the Complexity vs. Benefit Trade-off: The proposed H-HOUDINI algorithm is significantly more complex than classic HOUDINI. It requires white-box access to the design, a COI slicer, multiple oracles, and management of a parallel, recursive search state with memoization.

However, the benefit overwhelmingly justifies this complexity. The results presented are not incremental; they are transformative for the problem domain. * A 2880x speedup on Rocketchip (from 8 hours to <10 seconds) is a step-function improvement. * The ability to verify BOOM variants—a task where the prior MLIS-based art (CONJUNCT) failed to scale at all—demonstrates that H-HOUDINI breaks a fundamental scalability barrier.

This is a clear case where a more complex, novel algorithmic structure unlocks a new level of capability. The marginal gains critique does not apply here; the gains are substantial and qualitative.

---

Review Form

Summary

This paper presents H-HOUDINI, a novel algorithm that significantly advances the state-of-the-art in automated invariant learning for hardware verification. The core contribution is a new method that synthesizes the strengths of MLIS (predicate abstraction, positive examples) and SAT-based incremental learning (relative induction). The algorithm replaces expensive, monolithic SMT queries with a hierarchical wavefront of smaller, independent, and parallelizable relative inductivity checks. This is achieved by recursively applying abductive reasoning guided by the cone-of-influence of target predicates. The authors instantiate H-HOUDINI as VELOCT and demonstrate its power by learning security invariants for the complex RISC-V BOOM out-of-order processor, a task that was intractable for prior MLIS-based techniques.

Strengths

1. Novel Algorithmic Synthesis: The primary strength is the concrete and scalable algorithm that successfully instantiates the high-level concept of a Relative ICE (RICE) learner. The hierarchical decomposition based on the design's COI is a principled and effective method for breaking down a monolithic verification problem.
2. Demonstrated Step-Function Scalability: The work does not present a minor improvement. By scaling to the BOOM processor family, it solves a problem that was out of reach for prior art in this domain. The 2880x speedup on Rocketchip further underscores the significance of the algorithmic advance.
3. Principled Design: The algorithm is sound by construction and its design choices (e.g., use of positive examples to prune search, memoization, parallelism) are well-justified and contribute directly to its performance.

Weaknesses

1. Positioning of Novelty: The paper's framing could more clearly distinguish its contribution from prior conceptual work. The core idea of using relative induction in an MLIS/ICE context was previously proposed by Vizel et al. [48]. While the authors correctly cite this in the related work, the introduction could be more precise by stating that H-HOUDINI's novelty lies in being the first scalable, hierarchical algorithm to realize this concept, rather than the invention of the concept itself.
2. Dependence on Oracles and Annotations: The practical instantiation, VELOCT, relies on several oracles and, for the complex BOOM core, a modest number of expert annotations (Section 6.2, page 9). While the core search algorithm is the main contribution, this reliance on domain-specific configuration slightly detracts from the novelty of a fully automated, "push-button" solution.

Questions to Address In Rebuttal

1. Could the authors more explicitly delineate the novel algorithmic contributions of H-HOUDINI beyond the initial RICE-learning concept proposed by Vizel et al. [48]? Specifically, is the primary contribution the COI-based decomposition, the recursive abduction strategy, or the parallel/memoized implementation of the search?
2. The hierarchical decomposition seems tightly coupled to the 1-step COI of a hardware transition system. How does the core H-HOUDINI algorithm generalize to other domains, such as software verification, where the notion of a static, 1-step dependency graph is less clear or may be much larger? Does the novelty hold if the slicing oracle is less precise?
3. The paper presents a specific SMT query for the abduction oracle O_abduct (Section 3.2.3, page 6) based on UNSAT cores. How critical is this specific formulation to the algorithm's success? Could it be replaced with other interpolation or abduction techniques without fundamentally changing the performance characteristics of the overall H-HOUDINI search?

Paper Title: Instruction-Aware Cooperative TLB and Cache Replacement Policies Reviewer: The Guardian (Adversarial Skeptic)

---

Summary

The authors present a pair of cooperative replacement policies, iTP for the STLB and xPTP for the L2 cache, designed to mitigate performance degradation from instruction translation overheads in modern server workloads. The central thesis is that prioritizing instruction translations in the STLB (via iTP) is highly beneficial, but creates downstream pressure on the cache hierarchy due to an increase in data page walks. The proposed L2C policy, xPTP, is designed to specifically alleviate this pressure by preferentially retaining data page table entries (PTEs). The combined iTP+xPTP proposal includes an adaptive mechanism to toggle xPTP based on STLB pressure. The authors claim a significant 18.9% single-core performance improvement over an LRU baseline and superiority over several state-of-the-art policies.

Strengths

1. Clear Motivation: The paper effectively establishes the problem of instruction translation overhead in server workloads with large code footprints. The motivational studies in Section 3 (Figures 1, 3, 4) logically build the case for why a specialized, instruction-aware policy might be needed and correctly identify the negative side-effect (increased data page walks) that their cooperative policy aims to solve.

2. Logical Core Concept: The fundamental idea of cooperatively managing the STLB and a lower-level cache is sound. Recognizing that an aggressive STLB policy has consequences for the cache hierarchy and designing a second policy to explicitly mitigate those consequences is a logical approach.

3. Extensive Evaluation Space: The authors evaluate their proposal across a respectable number of configurations, including single-thread, 2-thread SMT, different LLC replacement policies (LRU, SHiP, Mockingjay), varying ITLB sizes, and multiple page sizes. This demonstrates a commitment to thoroughly testing the proposal's robustness.

Weaknesses

My primary concerns with this work center on the potential for selection bias in the evaluation, the ad-hoc nature of the policy design, and an incomplete analysis of the policy's negative trade-offs.

1. Workload Selection Bias: The paper's headline claims are derived from a curated set of 120 workloads selected specifically because they have an STLB MPKI of at least 1.0 (Section 5.2). This constitutes a significant selection bias. While useful for demonstrating the policy's potential in worst-case scenarios, it provides no insight into its performance on a more representative, un-filtered distribution of server workloads. The reported geomean improvements are likely inflated as they are calculated only across workloads predisposed to benefit. The work lacks an analysis of performance on workloads with low-to-moderate STLB pressure, where the policy might be neutral or even detrimental.

2. Arbitrary Policy Parameters and Lack of Sensitivity Analysis: The proposed policies, iTP and xPTP, are governed by a set of "magic numbers" (N=4, M=8 for iTP; K=8 for xPTP) that are presented as fixed values derived from "parameter space exploration" (Section 5.1). The paper provides no data from this exploration. This is insufficient. A rigorous work must demonstrate how these parameters were chosen and, more importantly, how sensitive the final performance is to their values. Without this analysis, the policies appear brittle and potentially over-fitted to the specific workloads and architecture under evaluation. For instance, why is a 3-bit frequency counter for iTP sufficient and optimal?

3. Adaptive Mechanism as an Admission of Harm: The introduction of an adaptive mechanism to disable xPTP during periods of low STLB pressure (Section 4.3.1) is a strong signal that xPTP can be actively harmful. The paper frames this positively as "Phase Adaptability," but fails to provide a crucial analysis of this behavior. It is essential to quantify the performance degradation caused by xPTP that necessitates this mechanism. The current design simply avoids the harm rather than analyzing its root cause.

4. Misleading Baseline for Headline Claim: The abstract and results prominently feature an 18.9% improvement. However, this is relative to a pure LRU baseline in both the STLB and L2C. LRU is a notoriously weak baseline for modern cache replacement. When compared to more realistic state-of-the-art policies like TDRRIP (Figure 8a), the improvement of iTP+xPTP is a more modest ~9.6% (18.9% vs 9.3%). While still significant, using the LRU comparison for the headline claim is misleading.

5. Incomplete Analysis of Cache Pressure: Figure 9a clearly shows that iTP+xPTP substantially increases the L2C MPKI (from 30.6 to 46.5 in the single-thread case). The authors argue this is compensated for by a reduction in LLC MPKI. However, this is a significant architectural trade-off that is not fully explored. This increased L2C-to-LLC traffic could become a new system bottleneck by consuming interconnect bandwidth and polluting the LLC with PTEs that could have been filtered at the L2. The implications for multi-core scalability beyond a simple 2-thread SMT are not addressed.

Questions to Address In Rebuttal

1. Please provide performance data (geomean and distribution) for your proposal on a complete, un-filtered set of server workloads from your source suite, not just those with STLB MPKI > 1.0. How does iTP+xPTP perform on workloads that do not heavily stress the STLB?

2. Can you provide data from your "parameter space exploration" for N, M, and K? Specifically, please include a sensitivity analysis showing how performance changes as these key parameters are varied from their chosen optimal values.

3. Please characterize the execution phases where the adaptive mechanism disables xPTP. What is the average performance loss incurred by running with xPTP enabled during these phases compared to an LRU policy?

4. The L2C MPKI increases by over 50% in the single-thread case when moving from LRU to iTP+xPTP. Can you discuss the potential system-level impact of this increased traffic on the LLC and memory interconnect, particularly in a many-core system where this effect would be amplified?

5. The violin plot in Figure 8a shows significant variance, with many workloads clustering near or below the performance of competing policies like TDRRIP and PTP. Can you provide a characterization of the workloads that do not benefit significantly from iTP+xPTP and explain why your mechanism is ineffective for them?

Paper Title: Instruction-Aware Cooperative TLB and Cache Replacement Policies Reviewer Persona: The Synthesizer (Contextual Analyst)

---

Summary

This paper identifies and addresses a critical performance bottleneck in modern server applications: pipeline stalls caused by instruction-stream misses in the last-level TLB (STLB). The authors argue that while the field has focused on data translation overheads, the large instruction footprints of contemporary workloads make instruction TLB misses particularly harmful.

The core contribution is a pair of cooperative replacement policies designed to work in synergy. The first, Instruction Translation Prioritization (iTP), is an STLB policy that aggressively prioritizes keeping instruction translations resident, accepting an increase in data translation misses as a trade-off. The second, extended Page Table Prioritization (xPTP), is a complementary L2 cache (L2C) policy designed to mitigate the negative side-effect of iTP. It does so by preferentially retaining cache blocks containing data Page Table Entries (PTEs), thereby servicing the increased data page walks from the L2C instead of main memory. The combined iTP+xPTP scheme is adaptive, enabling xPTP only when STLB pressure is high.

Through detailed simulation, the authors demonstrate that iTP+xPTP yields significant geomean performance improvements of 18.9% in single-threaded scenarios and 11.4% in SMT scenarios over a baseline LRU system, outperforming existing state-of-the-art TLB and cache replacement policies.

Strengths

1. Excellent Problem Formulation and Motivation: The paper does a superb job of contextualizing its work. The analysis in Section 3 (p. 3-5), particularly Figures 1 and 3, provides clear and compelling evidence that instruction address translation is a major, and often overlooked, performance limiter for the target class of server workloads. This immediately establishes the relevance and timeliness of the research.

2. Novel and Elegant Cooperative Design: The central idea of iTP+xPTP is its most significant strength. Rather than proposing two independent improvements, the authors have designed a holistic system. They correctly identify that aggressively optimizing one component (the STLB via iTP) creates a new pressure point elsewhere (the L2C via increased data page walks) and then propose a targeted solution (xPTP) for that specific side-effect. This demonstrates a deep understanding of microarchitectural interplay and represents a sophisticated approach to system design that is often missing in papers that focus on isolated components.

3. Strong Connection to Architectural Trends: The work is firmly grounded in the reality of modern system design. The problem of ever-growing instruction footprints in datacenter applications is well-documented. By focusing on the front-end stalls caused by instruction-fetch hazards, the paper addresses a problem that is not only current but is projected to worsen, ensuring the long-term relevance of the proposed solutions.

4. Comprehensive and Rigorous Evaluation: The experimental campaign is thorough. The authors evaluate their proposals not only in single-core and SMT configurations but also test their sensitivity to different ITLB sizes (Section 6.4, p. 11), the presence of large pages (Section 6.5, p. 11), and the use of different state-of-the-art LLC replacement policies (Section 6.3, p. 10). This comprehensive approach builds significant confidence in the robustness and general applicability of their findings. The reported performance gains are substantial and highly compelling.

Weaknesses

While this is a strong paper, there are opportunities to further contextualize and strengthen the work:

1. Limited Engagement with Instruction-Aware Cache Policies: The related work (Section 7, p. 13) correctly identifies instruction-aware cache replacement policies like Emissary [57] and CLIP [33]. The authors claim their work is orthogonal because xPTP is only concerned with data-PTEs, not instruction payload blocks. While technically true, this feels like a missed opportunity for a deeper synthesis. The ultimate goal is to reduce front-end stalls. A system using iTP+xPTP might still suffer from L2C misses on instruction code blocks. A truly state-of-the-art baseline would perhaps combine a policy like Emissary at the L2C/LLC with CHiRP at the STLB. A more insightful experiment would be to evaluate iTP+xPTP combined with Emissary to see if the benefits are additive, demonstrating a more complete, instruction-aware memory hierarchy.

2. Depth of Hardware Complexity Analysis: The overhead analysis in Section 4.1.3 and 4.2 (p. 6) is adequate but brief. The eviction logic for xPTP (Figure 6, p. 6) involves identifying an "alternative" LRU victim from a subset of blocks, which seems more complex than a standard LRU update. While this is unlikely to be on the critical path of an L2 hit, a more detailed discussion of the selection logic's timing and area implications would add another layer of practical credibility to the proposal.

3. Clarity on Parameter Tuning: The paper states that key parameters (N, M for iTP; K for xPTP) were determined via parameter space exploration (Section 5.1, p. 8). While it is noted that K has the highest impact, the paper would benefit from a small sensitivity analysis showing how performance varies with different values of K. This would help readers understand the tuning stability of the xPTP policy and whether the chosen value is a sharp peak or on a relatively flat plateau.

Questions to Address In Rebuttal

1. The core insight of your work is the synergy between TLB and cache policies. Could you elaborate on the potential synergy (or conflict) between your iTP+xPTP scheme and state-of-the-art instruction-aware L2/LLC cache replacement policies like Emissary [57]? Would combining them lead to further gains, or would they compete for the same resources in a detrimental way?

2. Could you provide more detail on the implementation complexity of the xPTP eviction policy? Specifically, can the "find ALT_LRU Victim" step (Figure 6b, p. 6) be performed in parallel with the standard LRU victim identification without impacting L2 miss latency?

3. Your adaptive mechanism for enabling xPTP is based on the STLB MPKI (Section 4.3.1, p. 7). Have you considered the impact of phase behavior? Could rapidly changing phases cause the mechanism to oscillate or lag behind, and how robust is the 1000-instruction evaluation window to such behavior?

Overall Recommendation: This is a high-quality paper with a novel, well-motivated, and impactful core idea. It addresses a significant, real-world problem with an elegant, cooperative solution backed by a strong evaluation. I recommend Accept. The weaknesses identified are primarily opportunities for strengthening the discussion and exploring future synergistic work, rather than fundamental flaws.

Paper Title: Instruction-Aware Cooperative TLB and Cache Replacement Policies Reviewer: The Innovator (Novelty Specialist)

---

Summary

This paper presents a pair of cooperative replacement policies, iTP for the STLB and xPTP for the L2 cache, designed to mitigate performance degradation from instruction translation misses in server workloads with large code footprints. The core idea is that iTP aggressively prioritizes instruction translations in the STLB, knowingly increasing page walks for data translations. The second policy, xPTP, is designed as a direct counter-measure, cooperatively prioritizing the page table entries (PTEs) for those data translations within the L2 cache to reduce the latency of the now-more-frequent data page walks. The authors claim this synergistic, instruction-aware design is novel and demonstrates significant performance improvements over state-of-the-art replacement policies.

My review focuses exclusively on the novelty of this contribution relative to the vast body of prior work on memory hierarchy management.

Strengths

The primary strength of this paper is the genuine novelty of its central thesis. Deconstructing the contribution reveals several distinct elements that, particularly in combination, represent a significant advancement over prior art.

1. Novelty of an Instruction-Aware STLB Replacement Policy (iTP): The concept of a replacement policy for a shared, last-level TLB that explicitly differentiates between instruction and data translations is, to my knowledge, new. Prior advanced STLB policies, such as CHiRP [55], are instruction-agnostic; they predict reuse based on control-flow history or other features but do not use the fundamental type of the memory access (instruction fetch vs. data load/store) as a primary signal. The motivation presented in Section 3, highlighting the distinct performance impact of instruction translation misses, provides a strong rationale for why this previously unexplored design space is worth investigating.

2. Novelty of the Cooperative "Problem/Solution" Mechanism: The synergy between iTP and xPTP is the most innovative aspect of the work. While cooperative hardware mechanisms are not new in principle, the design here is unique. iTP is designed to be "aggressively myopic"—it optimizes for instruction TLB hits at the direct and acknowledged cost of creating a new pressure point: data page walks. xPTP is not merely a generic translation-aware cache policy; it is purpose-built to alleviate the specific negative externality created by iTP. This explicit cause-and-effect relationship between policies in two different hierarchy levels (STLB and L2C) is a novel and elegant architectural pattern. It moves beyond policies that are simply "aware" of each other to a policy pair that is fundamentally symbiotic.

3. Clear Differentiation from Existing "Aware" Policies: The authors correctly identify the closest prior art and articulate the delta.

   • Unlike translation-aware cache policies like PTP [63] and TDR-RIP [79], the proposed iTP+xPTP scheme differentiates between instruction PTEs and data PTEs across the STLB/L2C boundary. PTP/TDR-RIP treat all PTEs monolithically.
   • Unlike instruction-aware cache policies like Emissary [57] or CLIP [33], which prioritize instruction code blocks, this work operates on instruction translations in the TLB and uses the cache policy (xPTP) to manage data PTEs, not code blocks. This is a crucial and novel distinction.

Weaknesses

From a novelty standpoint, the weaknesses are minor and relate more to the implementation details than the core concept.

1. Component-Level Mechanisms are Derivative: While the application of the policy is novel, the underlying mechanisms within iTP are not. The use of frequency counters (the Freq field in Section 4.1) and a differentiated insertion policy (inserting new instruction entries at MRUpos – N as described in Section 4.1.1, page 6) are established techniques in the broader cache replacement literature. The novelty here stems entirely from applying these techniques based on the instruction/data type trigger within the STLB, not from inventing a new method of recency stack manipulation. The paper would be stronger if it explicitly acknowledged that it is adapting well-known policy primitives for a new purpose.

2. Limited Exploration of Alternative Cooperative Designs: The paper presents the iTP+xPTP pairing as the solution. However, it does not explore whether other, perhaps simpler, cooperative schemes could achieve similar benefits. For instance, could iTP be paired with an existing instruction-aware cache policy like Emissary [57]? While the authors’ design choice seems logical, the lack of discussion of alternative cooperative pairings leaves the uniqueness of the xPTP component slightly less defended than it could be. The innovation is clear, but its necessity over other potential combinations is assumed rather than proven.

Questions to Address In Rebuttal

1. The core novelty of this work appears to be the synergistic combination and the specific targeting of instruction translations in the STLB. Could the authors please comment on the novelty of the iTP promotion/insertion mechanism itself? Setting aside its instruction-aware trigger, how does the manipulation of the LRU stack (using N, M, and a frequency counter) differ conceptually from prior predictive or frequency-based replacement policies in the cache domain?

2. The paper makes a compelling case for the iTP+xPTP pairing. However, a potential alternative could be to combine iTP (in the STLB) with a state-of-the-art instruction-aware cache policy like Emissary [57] (in the L2C), which prioritizes critical code blocks. This would help instruction fetches that miss in L1I but might do little for the data page walk problem. Could the authors elaborate on why their proposed cooperative structure—where the L2C policy compensates for a data-side weakness introduced by the STLB policy—is fundamentally superior to a structure where both policies synergistically target the instruction-side bottleneck? This would help solidify the novelty and rationale behind the specific design of xPTP.

Paper Title: Marionette: A RowHammer Attack via Row Coupling Reviewer: The Guardian (Adversarial Skeptic)

---

Summary

This paper introduces "Marionette," a new class of RowHammer attack that leverages a "coupled-row" phenomenon present in certain DRAM chips. The core idea is that activating one DRAM row (visible to the host) can simultaneously activate a second, physically distant row. The authors claim this mechanism can be used to bypass two major classes of software-based RowHammer defenses: tracking-based (e.g., SoftTRR) and isolation-based (e.g., Siloz). They demonstrate an end-to-end exploit that successfully bypasses a SoftTRR-protected system to achieve privilege escalation. They also claim their technique can enhance the success rate of a conventional attack on a bare-metal system by a factor of 1.66x.

However, the paper suffers from significant limitations regarding the timeliness of the underlying vulnerability, makes unsubstantiated claims against a key class of defense, and presents findings whose generalizability and practical impact on modern systems are questionable. While the initial characterization of the coupled-row phenomenon is sound, the leap to broad security implications is not fully supported by the evidence provided.

Strengths

1. Thorough Characterization of Coupled-Row Hammering: The work presented in Section 4.1 provides a rigorous and convincing comparison between conventional RowHammer and coupled-row-based hammering. The data in Table 5 and Table 6, showing bitflip location overlap ratios consistently above 95% and relative Bit Error Rates (BER) close to 1.0, effectively establishes that a coupled row acts as a near-perfect proxy for its partner in terms of inducing bitflips. This is a solid piece of foundational analysis.

2. Demonstrated Exploit Against a Tracking-Based Defense: The successful end-to-end exploit against SoftTRR (detailed in Section 8.2) is a non-trivial contribution. It provides a concrete proof-of-concept that the proposed attack vector can, under the right conditions, bypass a state-of-the-art software tracker by hammering a seemingly unrelated row that is not being monitored by the defense.

3. Clear Presentation: The paper is well-structured and clearly written. The background, methodology, and attack flow are explained logically, making the core concepts easy to follow.

Weaknesses

1. Limited Relevance and Timeliness of the Vulnerability: The paper's entire premise rests on a hardware vulnerability that appears to be a historical artifact. The authors themselves concede that they "could not find a coupled row in DIMMs manufactured after 2019" (Section 3.1, page 5). This is reiterated in the discussion (Section 10, page 11). This fact critically undermines the practical impact and relevance of the work. The paper is effectively demonstrating an attack against legacy hardware. Without evidence of this phenomenon in modern or forthcoming DRAM, the contribution is largely academic.

2. Unsubstantiated and Speculative Claims Against Isolation Defenses: A major weakness is the unsubstantiated claim of bypassing isolation-based defenses like Siloz. The paper dedicates significant text to this possibility (Section 7.2, page 8), but the authors ultimately admit, "we did not execute a RowHammer attack on Siloz" (Section 8, page 9). This is a critical failure of rigor. Presenting a purely theoretical attack vector against a major class of defense as a primary contribution, without any experimental validation, is unacceptable. All claims regarding the bypassing of Siloz are speculative and should be removed or heavily caveated as future work.

3. Questionable Generalizability of Coupled-Row Mapping: The paper's crucial finding that the PA's MSB defines the coupled-row pair (Section 5.2, page 7) is demonstrated on a limited set of specific Intel server systems. Modern memory controllers employ complex and often undocumented address interleaving and scrambling schemes. The paper provides insufficient evidence to conclude that this simple MSB relationship holds true across different vendors (e.g., AMD), different platforms, or even different BIOS configurations on the same platform. The attack's feasibility hinges on this predictable mapping, which may not be a general property.

4. Limited Scope and Realism of the End-to-End Exploit: The primary exploit demonstration is performed on an ECC-disabled Haswell system (System-a, Table 2). This is a decade-old architecture in a security-permissive configuration that is not representative of modern production servers, which almost universally employ ECC. Furthermore, while the claimed 1.66x "enhancement" to a conventional attack (from 6.7% to 11.1% success rate in S4, Figure 8) is a measurable increase, the absolute success rates remain low and depend on a 14-minute page table spraying phase (S3). The practical significance of this enhancement is debatable.

5. Oversimplification of In-DRAM TRR Analysis: The reverse-engineering of the counter-based in-DRAM TRR (Section 4.2, page 6) concludes that coupled rows share a single tracker entry ("Case 1" in Figure 4). This conclusion is based on indirect evidence (the absence of bitflips in a multi-row hammering test). While plausible, it does not definitively rule out other complex mitigation behaviors. The quick dismissal of sampling-based TRRs is also cursory; a probabilistic defense might be affected differently by the increased victim count from a coupled-row attack, a nuance not explored here.

Questions to Address In Rebuttal

1. Given your own findings that coupled rows are absent in post-2019 DRAM, please provide a compelling argument for the forward-looking relevance of this work. Why should the security community be concerned about a vulnerability that appears to have been unknowingly fixed by manufacturers years ago?

2. The claim that Marionette can bypass isolation-based defenses like Siloz is not supported by any experiment. You must either provide concrete data demonstrating a successful cross-VM or host-VM attack in a Siloz-like environment or remove these claims entirely from the paper. Speculation is not a substitute for evidence.

3. Can you provide stronger evidence that the physical address MSB consistently maps to the coupled-row bit across a wider and more diverse range of systems (e.g., different CPU vendors, DIMM configurations, motherboards)? How sensitive is this mapping to BIOS settings for memory interleaving?

4. Regarding the successful bypass of SoftTRR, can you provide more quantitative details beyond a binary success/failure outcome? For instance, what was the required hammer count, and how does this compare to HCfirst values measured on the FPGA?

5. Please justify the use of an ECC-disabled system for your primary exploit. How do you expect the attack's success rate and feasibility to change on a modern, ECC-enabled server, where single-bit flips are corrected and multi-bit flips within a word are required?

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper introduces "Marionette," a novel and elegant RowHammer attack vector that weaponizes the "coupled-row" phenomenon present in certain DRAM modules. The core insight is that in these modules, two physically distant DRAM rows are connected to the same wordline, causing them to be activated simultaneously. This behavior is transparent to the processor and operating system, which see two distinct row addresses. The authors demonstrate that this architectural curiosity is a significant security vulnerability.

Marionette exploits this "row coupling" to bypass the fundamental assumption of physical adjacency that underpins entire classes of software-based RowHammer defenses. By hammering an accessible row, an attacker can puppeteer its coupled partner inside a protected or isolated memory region, turning it into a "remote" aggressor. The paper provides a thorough characterization of coupled-row behavior, demonstrates a full end-to-end privilege escalation exploit that bypasses SoftTRR (a state-of-the-art tracking-based defense), and shows how the technique can significantly boost the success rate of conventional attacks. Finally, the authors propose practical modifications to existing software defenses to mitigate this new threat.

Strengths

1. Fundamental Contribution to the Field: The paper's primary strength lies in identifying and exploiting a vulnerability in the assumptions of the defense literature, not just an implementation flaw. The idea that RowHammer's effects are strictly local to an aggressor is a cornerstone of software mitigations. By demonstrating a practical way to violate this locality, this work forces a necessary re-evaluation of how we model and defend against RowHammer. It brilliantly connects a low-level circuit characteristic, previously noted in works like [44, 45], to a high-level security failure.

2. Clear and Powerful Attack Concept: The "Marionette" attack is conceptually clean and highly effective. The analogy of a puppet is perfectly suited and aids understanding. The core mechanism—hammering a row in user space to induce bitflips from a coupled row located in a protected region (e.g., adjacent to page tables)—is an elegant way to bypass defenses like SoftTRR that monitor accesses based on physical address proximity. The diagrams, particularly Figure 5 (page 8), are excellent at conveying this core concept.

3. Strong Empirical Validation: The authors provide compelling evidence to support their claims. The work is not merely theoretical. They begin with a careful, FPGA-based characterization to show that hammering a coupled row produces bitflips nearly identical in location and magnitude to hammering the aggressor row directly (Section 4.1, page 5-6). This establishes the foundational viability of the attack. The subsequent end-to-end exploit on a real server protected by SoftTRR (Section 8, page 9) provides undeniable proof of the attack's practicality and elevates the paper's impact significantly.

4. Forward-Looking and Constructive Mitigation Strategy: The paper does not simply present a new attack; it also charts a path forward. The proposed mitigations in Section 9 (page 11) are pragmatic and well-reasoned. The idea of exposing the coupled-row relationship via the DRAM module's SPD chip to the OS is a simple yet powerful hardware-software contract. This allows existing software defenses like Siloz and SoftTRR to be "patched" with awareness of these non-local relationships, rather than requiring a complete redesign. This constructive approach is a hallmark of high-quality systems security research.

Weaknesses

1. Limited Scope of Affected Hardware: The paper notes that the authors were unable to find coupled rows in DIMMs manufactured after 2019 (Section 3.2, page 5 and Section 10, page 11). While the authors rightly argue that the underlying circuit optimization could reappear, this finding somewhat limits the immediate, widespread impact of the attack on the newest generation of hardware. The work's primary relevance is therefore as a crucial lesson for future hardware designs and a threat to a significant, but aging, fleet of servers.

2. Conceptual Bypass of Isolation Defenses: While the end-to-end exploit against the tracking-based SoftTRR is a major strength, the bypass of the isolation-based Siloz is presented conceptually (Section 7.2, page 8). The logic is sound, and the authors are transparent about the complexity of demonstrating it in a multi-DIMM setup. However, the lack of an empirical demonstration makes this part of the contribution slightly less impactful than the successful SoftTRR attack.

Questions to Address In Rebuttal

1. Regarding the prevalence of coupled rows, the paper notes they were not found in post-2019 DIMMs. Could the authors elaborate on why they believe this might be the case? Is it possible that manufacturers have explicitly abandoned this design due to security concerns, or could the coupling mechanism have evolved to be harder to detect? A more detailed discussion on the long-term relevance of this phenomenon would strengthen the paper.

2. The bypass of Siloz is a compelling idea. Could the authors comment on the specific technical hurdles that prevented a practical demonstration? For instance, does identifying the complex address mappings in a multi-channel, multi-DIMM server pose an insurmountable barrier for an attacker in practice, or was this primarily an engineering effort constraint?

3. The paper's analysis suggests that future hardware defenses like PRAC, if implemented at the wordline-level, would be effective against Marionette (Section 10, page 12). Does this imply that any hardware defense that operates at the wordline granularity (e.g., counting activations, delaying accesses) would inherently mitigate this attack, simply because the two coupled rows are indistinguishable from that perspective? Clarifying this would help position Marionette within the broader context of the ongoing attack/defense co-evolution.

Paper Title: Marionette: A RowHammer Attack via Row Coupling Reviewer Persona: The Innovator (Novelty Specialist)

---

Summary

This paper introduces "Marionette," a RowHammer attack that leverages the recently discovered hardware phenomenon of "row coupling" in certain DRAM modules. The core idea is that activating one DRAM row (from the processor's perspective) simultaneously activates a second, physically distant row. The authors are not the first to discover this phenomenon, but they claim to be the first to weaponize it. They demonstrate that by hammering a row under their control, they can indirectly hammer its coupled-pair row, which may reside in a protected or monitored memory region. This technique is used to construct an attack that bypasses two major classes of software-based RowHammer defenses: tracking-based (e.g., SoftTRR) and, conceptually, isolation-based (e.g., Siloz). The authors provide an end-to-end demonstration of a privilege escalation exploit on a server protected by SoftTRR and quantify the attack's ability to enhance conventional RowHammer success rates.

Strengths

The primary strength of this paper lies in its successful translation of a known hardware artifact into a potent, demonstrated security attack vector. My evaluation of novelty is as follows:

1. Novel Application of a Known Phenomenon: The authors are transparent that the existence of coupled rows is not their discovery, properly citing prior work [28, 44, 45]. However, where prior work (notably [45] from members of the same group) characterized the phenomenon and only "briefly discussed" its exploit potential (Section 1, page 2), this paper provides the first complete, end-to-end weaponization. This leap from a hardware characterization study to a fully realized attack that bypasses state-of-the-art defenses is a significant and novel contribution in the security domain.

2. Novel Bypass Mechanism: The core mechanism of the Marionette attack—using a physically-linked but logically-separate row to evade software monitoring—is a novel instantiation of a stealthy attack. While the general concept of finding ways to trigger hardware faults without being monitored is not new, the use of row coupling for this specific purpose in the context of RowHammer is. It cleverly exploits the abstraction gap between the OS's view of memory (based on physical addresses) and the DRAM's internal physical reality.

3. Systematic Evaluation of the Novel Attack: The paper doesn't just propose the attack; it provides a systematic evaluation that validates its novelty. The characterization in Section 4.1, which shows that coupled-row hammering is nearly identical in effect to conventional hammering, is crucial work that establishes the new attack primitive as being as powerful as the original.

Weaknesses

My critique focuses exclusively on the boundaries of the paper's novelty and where the claims might overstate the conceptual advance.

1. Contribution is Application, Not Discovery: The most significant weakness, from a pure novelty standpoint, is that the foundational mechanism is not new. The paper's contribution is entirely contingent on the prior discovery of coupled rows. While the authors' application is novel, the work should be framed carefully as a security implication study of a known hardware feature, rather than the discovery of a new class of hardware vulnerability from first principles.

2. Conceptual vs. Demonstrated Novelty: The claim of bypassing isolation-based defenses like Siloz remains conceptual. As stated in Section 7.2 (page 9), "we did not execute a RowHammer attack on Siloz." While the reasoning for how a bypass would work is sound, the lack of a demonstration means this part of the claimed novel contribution is unsubstantiated. A truly novel work would have included this, given its importance.

3. Incremental Enhancement Claim: The contribution detailed in Section 8.3 ("Enhancing Conventional Attacks") feels incremental. Using coupled rows to double the number of victim rows for a given set of aggressor activations is a direct and somewhat obvious consequence of the row coupling phenomenon. While the 1.66x quantitative result is useful, the underlying idea is not a paradigm shift but rather an optimization of an existing attack.

Questions to Address In Rebuttal

1. Clarifying the Delta from Prior Art: The introduction (Section 1, page 2) states that the exploit possibility of a coupled row was "briefly discussed in prior work [45]." To precisely establish the novelty of this paper, could the authors please elaborate on the exact extent of this prior discussion? What specific attack vectors or defense bypasses, if any, were hypothesized in [45]? This is critical for the committee to understand the true delta of this work.

2. Generalizability of the Novel Mapping: The attack's feasibility seems to rely on a relatively simple mapping where the PA's MSB defines the coupled pair (Section 5.2, page 7). Is this a fundamental property of row coupling, or an artifact of the tested systems? How would the feasibility of identifying and exploiting coupled rows change if the coupling bit were interleaved in a more complex, proprietary address mapping scheme? The novelty of the attack is stronger if it is not dependent on this convenient mapping.

3. Longevity of the Novelty: The paper notes that coupled rows were not found in the DIMMs they tested that were manufactured after 2019 (Section 3.2, page 5). This suggests the core phenomenon enabling this novel attack may already be obsolete in new hardware. Is the primary contribution here the demonstration of an attack on legacy systems, or can the authors provide a compelling argument or evidence (e.g., from circuit design principles) that such cost-saving design choices are likely to reappear in future DRAM generations (e.g., DDR5/6 or HBM3/4)?

Paper: MEDUSA: Accelerating Serverless LLM Inference with Materialization Reviewer: The Guardian

---

Summary

The paper proposes MEDUSA, a system designed to reduce the cold start latency of serverless Large Language Model (LLM) inference. The authors correctly identify that two specific stages within the model loading phase—KV cache initialization and CUDA graph capturing—are major contributors to this latency. The core idea is to "materialize" the state required by these stages in an offline phase and restore it efficiently during the online cold start. To achieve this, the paper introduces two primary techniques: an "offline-online cooperated parameters restoration" method to handle non-deterministic data pointers in CUDA graphs, and a "triggering-kernels enhanced kernel address restoration" method to resolve randomized or hidden kernel addresses. The evaluation, conducted on 10 LLM models, claims to reduce model loading latency by 42.5% and the tail latency of time-to-first-token (TTFT) by 53.0% under simulated workloads.

Strengths

1. Problem Motivation: The paper does an excellent job of motivating the problem. The breakdown of the cold start timeline in Figure 1 (page 1) and across multiple models in Figure 2 (page 3) provides clear, quantitative evidence that KV cache initialization and CUDA graph capturing are significant bottlenecks, accounting for nearly 50% of the loading phase. This analysis is a valuable contribution in its own right.

2. Clear Identification of Core Challenges: The authors correctly identify the two most difficult technical hurdles to materializing CUDA graphs: the non-determinism of memory addresses for kernel parameters (Challenge I, page 5) and the randomized/hidden nature of kernel function addresses (Challenge II, page 5). The paper is structured around solving these specific, non-trivial problems.

Weaknesses

My primary concerns with this paper lie in the fragility of its core assumptions and the potential lack of generalizability of its proposed solutions. The techniques appear to be clever workarounds that may function for the specific set of models tested but lack the robustness required for a general-purpose system.

1. The Brittle Assumption of Deterministic Control Flow: The entire mechanism for restoring data pointers, "indirect index pointers" (Section 4, page 7), is predicated on the assumption that the host-side control flow, particularly the sequence of memory allocations (cudaMalloc), is perfectly deterministic across different process launches. While this may hold true for the simple, straight-line execution of the models tested, it is an extremely strong assumption that is unlikely to hold universally. Modern ML frameworks or complex model architectures can feature dynamic control flow, conditional memory allocations, or different execution paths based on configuration or even input shape properties. The paper acknowledges the need for validation (Section 4, page 7) but this simply confirms the brittleness; it does not solve it. A system that requires a full output comparison to validate its correctness for a given configuration is not a robust one. This weakness is relegated to the discussion (Section 8, page 12) but I see it as a fundamental flaw in the design.

2. The "Triggering-Kernels" Heuristic is Not a General Solution: The method for resolving hidden kernel addresses (Section 5, page 8) relies on another fragile assumption: that executing the first layer of a model is sufficient to force the CUDA driver to load all necessary modules for the entire model. The authors justify this by stating that LLM layers are structurally identical (Section 5.2, page 8). This is an oversimplification. This assumption fails for any model with heterogeneous architectures, such as Mixture-of-Experts (MoE) models where different layers may invoke different kernels, or models that fuse operations differently in initial or final layers. The technique feels more like a pattern-matching heuristic that works for a narrow class of standard Transformer models than a principled solution.

3. Unaddressed Practical Limitations: The work is explicitly limited to single-GPU models (Section 8, page 12). This is a significant limitation, as many state-of-the-art and production-grade LLMs require model parallelism and are served across multiple GPUs. By not addressing this, the paper's applicability to the most demanding and relevant LLM serving scenarios is questionable. Furthermore, the handling of device-side memory allocations is dismissed as a non-issue based on empirical analysis of 10 models. However, a single library update or a new custom kernel that utilizes device-side allocation could silently break the entire MEDUSA restoration process, leading to memory corruption or segmentation faults. A robust system cannot simply assume such behavior will never occur.

4. Inadequate Analysis of Failure Cases and Recovery: The paper does not discuss what happens when its assumptions are violated at runtime. If the memory allocation pattern changes, or if a required kernel module was not loaded by the "triggering-kernel," does the system crash? Does it fall back to the slow, traditional capturing path, negating its benefits and potentially violating service-level objectives? The lack of discussion on the operational robustness and fault tolerance of MEDUSA is a major omission.

Questions to Address In Rebuttal

1. On Determinism: The pointer restoration mechanism relies on a fixed memory allocation sequence. Can you provide a more rigorous argument for why this assumption is safe beyond the specific models tested? How would MEDUSA handle a framework (like a future version of PyTorch) that introduces optimizations like memory pre-allocation or a caching allocator that changes the sequence of underlying cudaMalloc calls? Does MEDUSA's mechanism have a fallback path if a pointer mismatch is detected during restoration, or does it lead to a fatal error?

2. On Kernel Address Restoration: Please address the generalizability of the "triggering-kernels" technique. Have you analyzed its effectiveness on models with heterogeneous layer structures, such as Mixture-of-Experts (e.g., Mixtral) or models with different attention mechanisms in different layers? What is the evidence that the kernels of the first layer are a superset of kernels in all subsequent layers for all architectures of interest?

3. On Device-Side Allocations: While you did not observe device-side allocations in your 10 test models, they are a standard feature of CUDA. How would MEDUSA detect that a kernel performs a device-side allocation, given that this happens without host-side API interception? Wouldn't this lead to an unresolvable pointer during restoration and subsequent memory corruption? Is it not a critical flaw that the system cannot guarantee correctness in the presence of such standard CUDA features?

4. On Multi-GPU Support: Can you elaborate on the fundamental challenges of extending this work to a multi-GPU setting (e.g., with tensor parallelism)? Is the problem simply constructing a cross-GPU index pointer table, or are there more complex issues related to inter-GPU communication primitives (e.g., NCCL calls) that are difficult or impossible to materialize and restore?

Review Form: Persona 2 - The Synthesizer (Contextual Analyst)

Summary

This paper, "MEDUSA: Accelerating Serverless LLM Inference with Materialization," addresses the critical problem of cold start latency in serverless environments, specifically for Large Language Models (LLMs). The authors correctly identify that beyond typical serverless overheads (like container startup), LLM inference introduces two new, substantial latency sources during the loading phase: KV cache initialization and CUDA graph capturing. These stages, while essential for high-throughput serving, involve expensive runtime profiling and construction, accounting for up to 50% of the loading phase latency (Figure 1, page 1).

The core contribution is an elegant "materialize-and-restore" approach. Instead of performing these steps dynamically at every cold start, MEDUSA performs them once in an offline phase. It materializes the necessary KV cache memory size and, more importantly, the fully constructed CUDA graphs. The technical novelty lies in the sophisticated techniques developed to restore the CUDA graphs, which are inherently stateful and non-portable due to hardcoded memory pointers and kernel addresses. To this end, the authors introduce an "offline-online cooperated parameters restoration" method using an intermediate representation (indirect index pointers) and a "triggering-kernels enhanced kernel address restoration" technique to resolve kernel addresses, even for closed-source libraries like cuBLAS. The result is a significant reduction in the loading phase and, consequently, a 53% reduction in tail time-to-first-token (TTFT) latency under real-world workloads.

Strengths

1. Excellent Problem Scoping and Motivation: The paper does a fantastic job of dissecting the LLM cold start problem and isolating the most significant new bottlenecks. Figure 1 on page 1 is a powerful motivator, clearly showing that KV cache initialization and CUDA graph capturing are not minor details but dominant factors. This precise problem identification elevates the work beyond generic cold start solutions.

2. Elegant Core Idea with Deep Technical Insight: The central concept of materializing application-level state is a highly effective specialization of the broader "checkpoint-and-restore" paradigm seen in systems like CRIU or FaaSnap [8]. Instead of a heavyweight, full-process snapshot, MEDUSA targets only the high-value, expensive-to-create state (the CUDA graphs). This is a much more lightweight and targeted approach perfectly suited for this domain. The recognition that the non-determinism of memory addresses is the key challenge, and the subsequent development of the indirect index pointer table (Section 4, page 7), is the paper's most significant technical strength. It is a clever, domain-specific solution to a problem analogous to address relocation in traditional program loaders.

3. Addresses a Critical and Timely Problem: With the proliferation of serverless LLM APIs from major cloud providers, optimizing the cold start experience is of immense practical and commercial importance. The TTFT is a crucial user-facing metric, and the bursty nature of inference requests makes the serverless paradigm highly attractive yet vulnerable to cold starts. This work is therefore situated at the confluence of several important research trends: serverless computing, systems for ML, and GPU optimization.

4. Strong and Relevant Evaluation: The evaluation is comprehensive, covering 10 popular LLMs of varying sizes. The comparison against a naive asynchronous baseline (vLLM + async) effectively demonstrates that simple parallelization is insufficient, strengthening the case for the authors' materialization approach. The use of the ShareGPT dataset for application traces ensures the results are representative of real-world conditions and validates the impressive reduction in tail latency.

Weaknesses

While the work is strong, its presentation and discussion could be strengthened by contextualizing its limitations more broadly.

1. Potential Fragility of Core Assumptions: The success of the indirect index pointer mechanism hinges on a strictly deterministic control flow for memory allocations. While this holds true for a given version of a model framework, it feels potentially fragile. Minor updates to PyTorch, CUDA drivers, or vLLM could alter the allocation sequence, invalidating the materialized state. The paper would benefit from a discussion on the robustness of this approach and the lifecycle management of the materialized artifacts (e.g., how often do they need to be regenerated?).

2. Limited Scope to Single-GPU: The discussion section (Section 8, page 12) acknowledges that the current implementation is for single-GPU models. This is a significant limitation, as many state-of-the-art and production models require multi-GPU serving via tensor or pipeline parallelism. Extending the pointer restoration and graph materialization concepts to a multi-GPU, multi-process environment is a non-trivial research challenge that is central to the work's future impact. This weakness should be positioned more prominently as a key direction for future work.

Questions to Address In Rebuttal

1. Robustness and Generalization: How sensitive is the materialized allocation trace to changes in the underlying software stack (e.g., a minor PyTorch or CUDA version bump)? Have you investigated what it would take to validate a materialized artifact against a given runtime environment to ensure compatibility before restoration?

2. Multi-GPU Serving: Could you elaborate on the fundamental challenges of extending MEDUSA to multi-GPU models? For instance, with tensor parallelism, inter-GPU communication operations (like all-reduce) are added to the graph. How would your materialization and restoration approach handle the pointers and state associated with these distributed operations?

3. Comparison with General-Purpose Snapshotting: What are the fundamental trade-offs between MEDUSA's application-specific materialization and a more general-purpose approach like using CRIU with GPU support (e.g., NVIDIA's CUDA-aware CRIU fork)? While MEDUSA is clearly more lightweight, a conceptual comparison of the overheads, restoration times, and flexibility of both approaches would better position your work in the broader systems landscape.

4. Triggering-Kernels Heuristic: The use of the first model layer as a "triggering-kernel" (Section 5.2, page 8) is a clever heuristic. Did you encounter any models or architectures where this heuristic failed, or where kernels needed for later layers were not loaded as part of the module for the first layer? This would speak to the generalizability of this specific technique.

Reviewer: The Innovator (Novelty Specialist)

Summary

This paper, "MEDUSA," targets the cold start latency problem in serverless Large Language Model (LLM) inference. The authors identify two major, yet overlooked, contributors to this latency: KV cache initialization and CUDA graph capturing. The core proposal is to mitigate this overhead through offline "state materialization," where CUDA graphs and KV cache metadata are generated once and then efficiently restored during subsequent cold starts. The authors claim novelty in two specific techniques designed to overcome the non-determinism inherent in this process: (1) an "offline-online cooperated parameters restoration" method that uses an "indirect index pointer table" to deterministically restore data pointers in the CUDA graph, and (2) a "triggering-kernels enhanced kernel address restoration" method to locate randomized and hidden kernel function addresses. The evaluation demonstrates a significant reduction in loading phase latency and a 53% reduction in tail time-to-first-token (TTFT) under a real-world workload.

Strengths

From the perspective of novelty, the paper's primary strength lies not in the high-level concept of state materialization, but in the specific, non-trivial mechanisms developed to make it feasible for CUDA graphs.

1. Novel Problem Decomposition: The paper correctly identifies that prior work on serverless cold starts (focused on container/runtime initialization) is insufficient for GPU-based LLM inference. The explicit identification and quantification of KV cache initialization and CUDA graph capturing as the dominant bottlenecks (Figure 1, page 1) is a valuable and novel insight that frames the problem effectively.

2. Novel Solution to Non-Deterministic Data Pointers: The core challenge with restoring a CUDA graph is that memory addresses (cudaMalloc) are non-deterministic across process launches. A blind memory dump is therefore useless. The proposed "indirect index pointer" (Section 4, page 7) is a genuinely novel approach to this problem in the context of CUDA. Instead of mapping old addresses to new addresses, it maps a pointer to its ordinal position in the deterministic sequence of allocation calls. Replaying this allocation sequence online allows for the perfect reconstruction of the pointer map. This is a clever and elegant solution that leverages the deterministic nature of the application's control flow to overcome the non-determinism of the underlying memory allocator.

3. Novel Solution to Hidden Kernel Addresses: The second major challenge is that kernel function addresses are also non-deterministic and, more problematically, sometimes hidden (e.g., cuBLAS kernels not exposed via dlsym). The "triggering-kernels" technique (Section 5, page 8) is another highly novel and pragmatic contribution. The insight that executing the first layer of an LLM is sufficient to force the CUDA driver to load all necessary kernel modules, which can then be introspected to find the required function pointers, is an inventive solution to a practical and frustrating problem for anyone working at this system level.

Weaknesses

My critique is centered on the framing of the novelty and its relationship to the vast body of prior art in checkpoint/restore (C/R) systems.

1. Understated Relationship to General C/R: The paper positions itself as "state materialization," which is semantically correct but downplays the fact that this is a highly specialized, application-aware form of checkpoint/restore. Decades of work exist on process C/R (e.g., CRIU, DMTCP) and more recently for serverless functions ([8, 54] cited by the authors). These systems solve the general problem of non-deterministic pointers and code addresses through mechanisms like page table manipulation and pointer relocation. The paper's novelty would be sharpened by more explicitly contrasting its semantic, lightweight approach with these general, heavyweight approaches in the introduction, rather than primarily in the related work section. The key innovation is avoiding a full process snapshot by understanding the semantics of a CUDA graph, and this point could be made more forcefully.

2. The "Indirect Index Pointer" is a Form of Relocation Map: The concept of re-linking pointers after a restore is not fundamentally new; it is a classic problem in serialization and C/R, often solved with relocation tables or "pointer swizzling." The novelty in MEDUSA is not the idea of re-mapping pointers, but the specific and efficient method for generating this relocation map: by tracking the allocation sequence rather than scanning and patching the entire memory space. This is a subtle but important distinction that should be clarified. The current phrasing might imply the entire concept of handling pointers is new, which is not the case.

Questions to Address In Rebuttal

1. The core assumption behind the "indirect index pointer" technique is that the sequence of buffer allocations is strictly deterministic. While this holds for the model's initialization path, could this assumption be violated by subtle changes in library versions (e.g., PyTorch, CUDA toolkit) or different hardware (e.g., different GPU architectures leading to different kernel choices and memory patterns)? How robust is this assumption in practice?

2. The "triggering-kernels" technique relies on running the first layer of the model to load necessary CUDA modules. Does this guarantee that all modules for all possible execution paths (e.g., for different batch sizes or sequence lengths captured in the offline graphs) are loaded? Could there be a case where a kernel needed for a batch size of 32 is in a module that is not loaded when only executing the first layer with a batch size of 1?

3. Could the authors compare their approach to a hypothetical one using a general-purpose C/R tool like CRIU with GPU support? My hypothesis is that CRIU would be far too slow and heavyweight, but explicitly arguing this point would further strengthen the case for MEDUSA's specialized, novel approach. Why is semantic materialization fundamentally better than a generic process snapshot for this specific problem?

Paper Title: METASAPIENS: Real-Time Neural Rendering with Efficiency-Aware Pruning and Accelerated Foveated Rendering Reviewer: The Guardian

---

Summary

The authors present METASAPIENS, a comprehensive system aimed at achieving real-time, high-quality Point-Based Neural Rendering (PBNR) on mobile platforms. The work is composed of three primary contributions: (1) An "efficiency-aware" pruning technique that moves beyond simple point counting to a metric based on computational cost versus visual contribution; (2) A foveated rendering (FR) pipeline for PBNR that uses a hierarchical point representation and HVS-guided training to reduce peripheral rendering load; and (3) A co-designed hardware accelerator that introduces mechanisms like tile merging and incremental pipelining to address the load imbalance issues inherent in PBNR, particularly when augmented with foveation. The authors claim an order of magnitude speedup over existing models with no loss of subjective visual quality, supported by a user study, objective metrics, and hardware synthesis results.

Strengths

1. Problem Formulation: The paper correctly identifies a key weakness in existing PBNR pruning literature: the disconnect between point count and actual computational cost. The analysis in Section 3.1 (page 4), particularly the correlation shown in Figure 4 between latency and tile-intersections rather than point count, is a solid and valuable observation that motivates the work well.

2. Principled Approach to Foveation: The integration of an established perceptual metric (HVSQ) directly into the training and pruning loop for the different foveal levels (Section 4.3, page 7) is a principled approach. This is superior to using ad-hoc heuristics like simple blurring or random subsampling for quality degradation in the periphery.

3. Systems-Level Scope: The work is ambitious in its scope, addressing the problem across the full stack from rendering algorithms to custom hardware. This holistic perspective is appropriate for a top-tier systems conference and demonstrates a thorough consideration of the problem.

Weaknesses

My primary concerns with this work center on the rigor of the proposed metrics, the justification for key design choices, and the robustness of the evaluation, particularly the user study.

1. Ambiguity and Heuristics in the Pruning Metric: The proposed "Computational Efficiency" (CE) metric (Section 3.2, page 4) is not as robustly defined as it needs to be.

   • The Val_i term, defined as the number of pixels "dominated" by a point, is ambiguous. In volume rendering with semi-transparent Gaussians, "domination" is not a binary concept. A pixel's final color is a blend of contributions. How are ties or near-ties handled? This definition seems unstable and could lead to inconsistent pruning results based on minor floating-point variations.
   • The use of the maximum CE across all training poses to characterize a point is a questionable choice. This makes the metric highly sensitive to outliers. A point that is useful in 99% of views but has a low CE in a single, unusual camera pose could be unfairly targeted for pruning. A justification for this choice over a more robust statistical measure (e.g., mean, median, or 90th percentile) is absent.
   • The overall iterative process in Figure 6 (page 5) appears heuristic, relying on a fixed pruning percentage (R=10%) and retraining cycles. This lacks a deeper theoretical grounding.

2. Unsupported Claims in the User Study: The user study (Section 7.1, page 10) is the foundation for the paper's central claim of maintaining visual quality, yet it is critically flawed.

   • The sample size of 12 participants is insufficient to draw strong, generalizable conclusions about subjective preference, especially for a perceptual task. While common in some HCI contexts, for a claim as strong as "no-worse than" or even "preferred over" a state-of-the-art dense model, this is well below the standard for rigorous perceptual science.
   • The claim that METASAPIENS-H is subjectively better than the dense MINI-SPLATTING-D is extraordinary and requires extraordinary evidence. The provided explanation—that pruning removes points trained with "inconsistent information"—is a post-hoc rationalization. No direct evidence of these supposed artifacts (e.g., flickering, luminance shifts) in the baseline is presented in the paper. Without a controlled comparison showing these specific artifacts, this conclusion is unsubstantiated speculation.

3. Insufficient Detail in Hardware Comparison: The hardware comparison to GSCore (Section 7.5, page 12) lacks transparency.

   • The authors state they "proportionally scale both GSCore and ours based on their own resource ratio." Technology node scaling for architectural comparison is notoriously complex and prone to error. The paper provides no details on the tool or methodology used for this scaling. Was a standard tool like DeepScaleTool [63] used? Were logic, SRAM, and wire delays scaled differently? Without this information, the iso-area comparison in Figure 15 is not verifiable and cannot be considered rigorous.
   • The tile merging unit relies on a threshold β (page 8) to trigger a merge. The paper provides no information on how this critical parameter is determined, if it is scene-dependent, or its sensitivity. This makes the effectiveness of a key hardware contribution difficult to assess.

4. Selective Multi-Versioning Appears Ad-Hoc: The "Selective Multi-Versioning" concept (Section 4.2, page 7) feels like an admission that the strict subsetting data representation is too restrictive to maintain quality. The choice to multi-version only Opacity and SHDC is justified empirically, but this lacks a principled explanation. Why are these parameters more critical than, for example, scale or the higher-order SH components for quality preservation in the periphery? A sensitivity analysis or ablation study is needed here.

Questions to Address In Rebuttal

1. Please provide a precise, algorithmic definition of how a pixel is determined to be "dominated" by a point (Val_i), especially in cases of multiple, semi-transparent Gaussians contributing to its color. How are ties or near-equal contributions handled?

2. Please justify the design choice of using the maximum CE across training poses for pruning decisions, rather than a more robust statistical aggregator like the mean or a high percentile. Provide data showing this choice leads to better outcomes.

3. Regarding the user study, please provide direct evidence (e.g., side-by-side video clips in supplementary material) of the "incorrect luminance changes" and other visual artifacts you claim are present in the dense MINI-SPLATTING-D baseline and are resolved by your pruning method.

4. Please provide explicit details on the methodology and any tools used to perform the architectural scaling of the GSCore baseline for the iso-area performance comparison presented in Figure 15 (page 11).

5. Explain how the tile merging threshold β in the hardware accelerator is selected. Is this a fixed, empirically-derived value, or is it adapted based on scene statistics? How sensitive is the performance gain from tile merging to the choice of β?

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper presents METASAPIENS, a full-stack system designed to enable real-time, high-fidelity Point-Based Neural Rendering (PBNR), specifically targeting mobile and edge devices. The authors identify that existing PBNR methods like 3D Gaussian Splatting, while faster than NeRFs, are still too computationally intensive for real-time mobile performance.

The core contribution is a synergistic, three-part solution that spans algorithms, human perception, and hardware architecture: 1. Efficiency-Aware Pruning: A novel pruning methodology that optimizes directly for rendering cost (measured in tile-ellipse intersections) rather than the indirect and less effective metric of point count. 2. Foveated PBNR: The first foveated rendering framework specifically tailored for PBNR, which uses an elegant hierarchical point representation (sub-setting with selective multi-versioning) to minimize storage and computation overhead while relaxing rendering quality in the visual periphery. This process is guided by a formal Human Visual System (HVS) quality metric. 3. Accelerated Hardware: A co-designed hardware accelerator that introduces novel mechanisms (Tile Merging and Incremental Pipelining) to specifically address the severe workload imbalance introduced by foveated rendering, a critical bottleneck that would otherwise nullify performance gains.

The authors demonstrate through extensive evaluation, including a user study, that their system achieves an order of magnitude speedup over existing PBNR models on a mobile GPU (and more with the accelerator) with statistically equivalent or better subjective visual quality.

Strengths

The primary strength of this paper is its outstanding holistic and principled approach. It is a quintessential systems paper that beautifully illustrates the power of co-design across multiple layers of abstraction.

1. Excellent Problem Connection and Framing: The paper correctly identifies a critical bottleneck for a very timely and impactful technology (neural rendering for AR/VR). Instead of proposing a narrow algorithmic tweak, the authors have diagnosed the problem from a systems perspective and proposed a comprehensive solution.

2. A Fundamental Shift in Pruning Philosophy: The insight presented in Section 3 (page 4) to move away from pruning based on point count towards pruning based on computational cost (tile intersections) is a significant contribution. This is a more direct and physically grounded optimization target. The "Computational Efficiency" (CE) metric is simple, intuitive, and demonstrably more effective than prior art. This idea has the potential to influence future work in optimizing not just PBNR, but other primitive-based rendering techniques.

3. Elegant and Efficient Foveated Rendering Design: Applying foveated rendering is not new, but the authors’ approach for PBNR is highly novel. The hierarchical point representation, where lower-quality models are strict subsets of higher-quality ones (Section 4.2, page 6), is a clever way to avoid the massive storage and redundant computation overhead of maintaining multiple independent models. The refinement of "Selective Multi-Versioning" is a pragmatic and effective engineering trade-off, allowing for quality tuning where it matters most (e.g., opacity) without sacrificing the efficiency of shared parameters.

4. True Algorithm-Architecture Co-design: The hardware accelerator design is not an afterthought; it directly addresses a critical performance problem created by their own algorithmic choice (foveated rendering). The load imbalance issue detailed in Section 5.2 (page 8) is a classic pipeline-killer, and the proposed solutions of Tile Merging and Incremental Pipelining are well-reasoned architectural techniques to solve it. This demonstrates a deep understanding of how software decisions impact hardware efficiency.

5. Strong Validation with a User Study: The inclusion of a psychophysical user study (Section 7.1, page 10) to validate that their optimizations do not compromise subjective visual quality is a major strength. It grounds their claims in human perception, which is the ultimate goal of foveated rendering, and elevates the work beyond simple PSNR/SSIM comparisons.

Weaknesses

While the work is very strong, there are a few areas where its context and limitations could be further explored. My comments here are not intended to detract from the core contribution, but to place it in an even broader context.

1. Training Complexity and Generalization: The proposed training pipeline (Figure 6, page 5, and Section 4.3, page 7) is an iterative process of pruning and retraining guided by the HVSQ metric. The authors note this increases training time roughly 3x. While this is a one-time offline cost per scene, it may become a practical barrier for applications requiring rapid or on-the-fly scene capture and optimization. The work positions itself as a rendering system, so this is a minor point, but it's worth acknowledging.

2. Static Nature of the Perceptual Model: The system relies on a fixed-ring model of eccentricity for foveation. This is standard practice, but the field of perceptual graphics is moving towards more dynamic models that might incorporate factors like scene content (e.g., saliency), task context, or even cognitive load. This work provides a fantastic foundation upon which such future, more dynamic foveation strategies for PBNR could be built.

3. System-Level Dependencies: As with all foveated rendering systems, the performance gains are predicated on the existence of a fast and accurate eye-tracker. The paper rightly uses hardware (Meta Quest Pro) that has one, but this dependency is a crucial practical constraint for deployment on the wider ecosystem of mobile devices.

Questions to Address In Rebuttal

1. The HVS-guided training is a cornerstone of the approach. Can the authors comment on its robustness? For example, are there specific scene types or rendering artifacts (e.g., temporal flickering of small, high-frequency details in the periphery) that the HVSQ metric might not fully capture, potentially leading to subjective quality degradation not caught by the user study's specific scenes?

2. Regarding the hardware accelerator, the design choices (e.g., 8 Culling Units, 16x16 VRC array) are balanced for the FR workload. How would this accelerator perform on a non-foveated, dense PBNR workload compared to a baseline like GSCore [39] that was optimized for it? This would help clarify the trade-offs made and whether the proposed architecture is specialized for FR or generally superior.

3. The concept of pruning based on "tile intersections" is powerful. Have the authors considered its applicability beyond PBNR? It seems this principle could extend to other rasterization-based techniques with variable primitive screen-space footprints, such as mesh rendering with complex shaders. A brief comment on the potential for broader impact would strengthen the paper's contribution.

Reviewer: The Innovator (Novelty Specialist)

---

Summary

This paper presents METASAPIENS, a system designed to achieve real-time Point-Based Neural Rendering (PBNR), specifically targeting mobile devices. The authors propose a three-pronged approach to accelerate the 3D Gaussian Splatting pipeline: (1) an "efficiency-aware" pruning method that prioritizes removing points based on their computational cost rather than just their number; (2) a foveated rendering (FR) technique tailored for PBNR that uses a hierarchical, subset-based point representation to reduce rendering load in the visual periphery; and (3) a co-designed hardware accelerator that introduces tile merging and incremental pipelining to mitigate load imbalance issues exacerbated by foveated rendering. The authors claim this is the first system to deliver real-time PBNR on mobile devices, with a user study confirming that the visual quality is comparable to a dense, state-of-the-art model.

---

Strengths

From a novelty perspective, the paper's primary strength lies in its specific formulation of the PBNR performance problem and the resulting pruning metric.

1. Novel Problem Formulation for Pruning: The authors correctly identify that raw point count is a poor proxy for computational cost in PBNR. The analysis in Section 3.1 and Figure 4 (page 5), which demonstrates that inference latency correlates with the number of tile-ellipse intersections rather than the point count, is a sharp and important insight for this domain.

2. Novel Pruning Metric: Building on this insight, the proposed Computational Efficiency (CE) metric (Section 3.2, page 4) is a direct and novel contribution. While cost-aware pruning is a known concept in the broader ML compression literature, its formulation here—quantifying cost as the number of intersected tiles—is specific to the PBNR rasterization pipeline and appears to be genuinely new. This is the most significant novel idea in the paper.

3. System-Level Synthesis: The paper does a commendable job of synthesizing techniques from disparate fields—perceptual science (HVSQ metric), traditional computer graphics (LOD-like structures), and hardware architecture (load balancing)—into a cohesive system for a modern rendering problem. While the novelty of individual components can be debated, their integration is non-trivial and represents a novel system design.

---

Weaknesses

My primary concern is that several of the core ideas presented as novel are, in fact, direct applications or re-discoveries of well-established concepts from traditional computer graphics and hardware architecture. The novelty lies in their application to PBNR, but the underlying concepts themselves are not new.

1. Foveated Rendering Approach Lacks Conceptual Novelty: The paper claims to "introduce the first FR method for PBNR" (Section 1, page 2). However, the core data structure enabling this—where points for lower-quality regions are a strict subset of those for higher-quality regions (Section 4.2, Figure 7C, page 6)—is conceptually identical to classic Level-of-Detail (LOD) hierarchies used in computer graphics for decades. Techniques like progressive meshes or hierarchical point representations (e.g., QSplat [62]) are built on the exact same principle of creating coarser representations by simplifying or sub-sampling finer ones. Applying this standard LOD management strategy to a new primitive (3D Gaussians) for the purpose of foveated rendering is an engineering adaptation, not the invention of a new FR method. The "selective multi-versioning" is an incremental refinement to this known strategy to trade storage for quality.

2. Architectural Contributions are Applications of Known Patterns: The hardware accelerator enhancements, while effective, are applications of standard architectural patterns for handling workload imbalance and streaming data.

   • Tile Merging (Section 5.2, page 8): This is a form of work coalescing or batching, a fundamental technique in parallel computing (especially GPUs) to improve utilization by grouping small, independent work items into larger, more efficient chunks. Its application here is well-motivated but does not represent a new architectural concept.
   • Incremental Pipelining with Line Buffers (Section 5.2, page 9): Line-buffering is a canonical technique in streaming image processing hardware used to manage producer-consumer dependencies with minimal on-chip storage, avoiding the need for a full tile/frame buffer between pipeline stages. Using it to enable sub-tile-level pipelining is a direct and textbook application of this pattern.

3. Perceptual Guidance is an Application of Prior Work: The training framework's use of the HVSQ metric (Section 4.3, page 7) is an application of the metric developed by Walton et al. [72]. While its integration to guide the generation of foveated PBNR models is a good use of this prior work, it should be framed as an application rather than a novel contribution in itself.

In summary, the paper's claims of novelty are overstated in several key areas. The work's primary contribution is the clever and effective adaptation of existing ideas to the specific domain of PBNR, rather than the introduction of fundamentally new algorithms or architectures.

---

Questions to Address In Rebuttal

1. Please clarify the novelty of the hierarchical, subset-based point representation (Section 4.2) in light of classical Level-of-Detail (LOD) techniques used in computer graphics for decades, which employ the same core principle. How does your method fundamentally differ from applying a standard LOD framework to 3D Gaussian primitives?

2. Could the authors position their architectural contributions (tile merging, incremental pipelining) relative to prior art in the broader field of parallel processor and accelerator design? Specifically, please discuss how these proposals differ from established techniques for work coalescing and streaming pipeline design.

3. The CE pruning metric (Section 3.2) is the paper's strongest novel contribution. To help situate it, is the general principle of pruning a model based on a direct measure of computational cost (vs. an indirect proxy like opacity or activation magnitude) a known concept in other domains? If so, please clarify that the novelty is specifically in the formulation of this cost for the PBNR pipeline.

Paper Title: MOAT: Securely Mitigating Rowhammer with Per-Row Activation Counters Reviewer: The Guardian (Adversarial Skeptic)

---

Summary

The authors present MOAT, a Rowhammer mitigation mechanism designed to be implemented within the JEDEC PRAC+ABO framework. The paper's claimed contributions are four-fold: (1) it introduces "Jailbreak," an attack that purportedly breaks the security of Panopticon, a foundational prior work; (2) it proposes MOAT, a dual-threshold, single-entry tracker as a secure alternative; (3) it analyzes the security implications of delayed ALERTs in the JEDEC specification via a "Ratchet Attack"; and (4) it evaluates the potential for performance-degradation attacks against its own mechanism.

While the paper identifies several interesting attack vectors and presents a seemingly pragmatic design, its central claims of security are not rigorously substantiated. The work overstates its contributions, particularly the notion of being "provably secure," and prematurely dismisses clear vulnerabilities in its own proposal. The analysis rests on analytical models whose assumptions are not sufficiently challenged, and the overall security guarantees are weaker than claimed.

Strengths

Despite my significant reservations, the paper does contain kernels of valuable insight:

1. The Deterministic Jailbreak Attack: The core insight in Section 3.2 (Page 5) that a simple FIFO queue without associated counter values is vulnerable to withholding a "youngest" entry from mitigation is a valid and important finding. It serves as a good cautionary tale for implementers of the PRAC+ABO framework.

2. Analysis of Inter-ALERT Activations: The "Ratchet Attack" detailed in Section 5.2 (Page 7) is the paper's strongest contribution. To my knowledge, this is the first work to systematically analyze how an attacker can weaponize the JEDEC specification's allowance for a minimum number of activations between ALERTs. The analysis correctly identifies that these seemingly innocuous activations can be leveraged to create an "amplification" effect, pushing row activation counts significantly beyond the ALERT threshold (ATH). Figure 9 (Page 8) provides a clear, albeit simplified, illustration of this principle.

Weaknesses

My review focuses on the correctness and rigor of the work. On these grounds, the paper has several critical flaws that undermine its conclusions.

1. The Claim of "Provably Secure" is Unfounded and Misleading: The abstract explicitly states MOAT is a "provably secure design." This is a profound overstatement. The security analysis relies entirely on the analytical model presented in Appendix A (Page 14). This is a model, not a proof. A formal proof would involve a rigorous framework (e.g., using theorem provers like Coq or Isabelle/HOL) with a precisely defined threat model and machine-checked verification of security properties. The provided model is a set of algebraic equations whose security guarantees are only as strong as its underlying assumptions about attacker behavior. The authors have not proven MOAT secure; they have shown that it is secure under their model. This is a critical distinction that the paper fails to make, which is unacceptable for a work on security.

2. Premature and Unjustified Dismissal of DoS Vulnerability: In Section 7.3 (Page 10), the authors analyze the Torrent-of-Staggered-ALERT (TSA) attack and find it can cause a 52% throughput loss. They then immediately dismiss this by claiming it is "not a serious new vulnerability" because it is "similar in range to other memory contention attacks, such as row-buffer conflicts." This reasoning is specious. The existence of one type of performance bottleneck does not excuse the introduction of a new, potent, and attacker-triggered one. A 52% degradation is a significant denial-of-service vector by any reasonable standard. The authors provide no evidence to support their claim that this is not a "serious" issue and are essentially hand-waving away a fundamental weakness in mechanisms that rely on stalling the memory controller.

3. The Randomized Jailbreak is an Impractical and Overstated Threat: The deterministic Jailbreak is a clear flaw in a naive Panopticon implementation. However, the "Randomized Jailbreak" (Section 3.3, Page 5) is far less convincing. The attack's success hinges on a probabilistic event with a 2⁻¹⁶ chance of success per attempt. While the authors calculate an average success time of 16 seconds, this belies the reality that many attempts could be required, making it noisy and potentially detectable. To present this on equal footing with the deterministic attack (as in Figure 5, Page 6) inflates the contribution and paints a misleading picture of the threat. Is a non-deterministic attack that requires an average of 16 seconds of specific memory patterns a practical threat that "breaks" the randomized defense? I argue it is not.

4. Unchallenged Threat Model Assumptions for Ratchet Attack: The analytical model for the Ratchet Attack assumes a perfect attacker with flawless control over timing. It assumes the ability to precisely schedule activations within the 180ns window before an RFM and between consecutive ALERTs. In a real system, factors such as OS scheduler jitter, non-deterministic memory controller reordering, cache contention, and other system noise would make achieving the perfect activation sequence described in the model exceedingly difficult. The paper makes no attempt to discuss the practical feasibility of this attack or how robust the attack is to timing perturbations. Without this analysis, the "Safe TRH" values derived in Figure 10 (Page 8) represent a theoretical worst-case that may be impossible to achieve in practice.

Questions to Address In Rebuttal

The authors must provide clear and direct answers to the following questions. Vague responses will be considered insufficient.

1. The term "provably secure" carries a specific and strong meaning in the security community, typically implying formal verification. Please provide the formal proof for MOAT. If one does not exist and the claim is based solely on the analytical model in Appendix A, you must justify this use of terminology and explicitly state all assumptions under which your security claims hold.

2. Provide a rigorous argument for why a 52% attacker-induced performance degradation, as demonstrated by the TSA attack, should not be considered a significant Denial-of-Service vulnerability. On what basis do you conclude this is an acceptable risk?

3. How does the effectiveness of the Ratchet Attack degrade in the presence of realistic system timing noise? Provide sensitivity analysis or a reasoned argument about the timing margins an attacker has to successfully execute the activation patterns required by your model.

4. Regarding the MOAT-RP extension (Section 9, Page 12), the "Tardiness Damage (TD)" value of 20 activations appears to be an ad-hoc parameter. How was this value derived? What analysis was done to ensure an attacker cannot induce a Tardiness Damage greater than 20 before an ALERT is triggered?

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper addresses the critical and timely problem of implementing secure Rowhammer mitigations within the new JEDEC DDR5 PRAC+ABO framework. The authors make a compelling case that this industry standard, while a significant step forward, is merely a framework whose security is contingent on its implementation. The work's central narrative is built on two pillars: first, a novel and effective "Jailbreak" attack that breaks Panopticon, the academic proposal that inspired PRAC+ABO. Second, the paper proposes MOAT, a low-overhead and provably secure design that instantiates the PRAC+ABO framework correctly.

MOAT's design is elegant in its simplicity, using dual thresholds (ETH for eligibility, ATH for alerts) and a minimalist single-entry-per-bank tracker. The authors go beyond a simple proposal, providing a deep security analysis of subtle vulnerabilities in the ABO protocol itself, leading to their novel "Ratchet Attack." They also thoroughly evaluate performance overheads and resilience to Denial-of-Service attacks. The core contribution is not just a new mechanism, but a comprehensive blueprint for securely navigating the design space opened up by next-generation DRAM standards.

Strengths

1. Exceptional Timeliness and Relevance: This work is positioned perfectly at the intersection of academic hardware security research and real-world industry standards. By directly engaging with the JEDEC PRAC+ABO specification, the paper provides invaluable and immediate guidance to DRAM vendors and system architects. This is not a solution in search of a problem; it is a direct and necessary investigation of a critical emerging technology.

2. Powerful Motivating Attack: The "Jailbreak" attack on Panopticon (Section 3, page 5) is a significant contribution in its own right. By demonstrating a fundamental flaw in the logical predecessor to the JEDEC standard (specifically, the use of a simple FIFO queue without storing counter values), the authors establish a clear and urgent need for a more principled implementation. This negative result is as important as the positive proposal of MOAT.

3. Insightful Security Analysis: The paper's analysis of "Delayed ALERTs" and the resulting "Ratchet Attack" (Section 5, page 7) is a standout feature. It reveals a subtle but exploitable aspect of the JEDEC specification itself, showing that the small number of activations permitted between ALERTs can be weaponized to bypass the intended threshold. This level of deep protocol analysis elevates the paper beyond a simple mechanism proposal and provides a lasting contribution to our understanding of the problem.

4. Comprehensive and Practical Design: MOAT is not just secure, it is practical. The design's extremely low overheads (0.27% slowdown and 7 bytes of SRAM per bank, as stated in the abstract) make it a viable candidate for real-world deployment. The evaluation is thorough, considering not only security and benign-case performance but also robustness against performance-degradation attacks like the proposed "Torrent-of-Staggered-ALERT" (TSA) attack (Section 7.3, page 10).

Weaknesses

While the paper is strong, there are areas where the contextualization could be broadened.

1. Limited Exploration of the Design Space: The paper strongly argues for the superiority of a single-entry tracker (MOAT-L1). While the reasoning is sound (lower overhead, smaller attack surface), the discussion could benefit from a more nuanced exploration of why a designer might ever choose a higher ABO level. Are there potential system-level benefits to batching mitigations (e.g., interaction with power management, simplified MC scheduling) that would make a multi-entry tracker desirable despite the security trade-offs? The paper shows that higher levels are worse for security and performance, but not why they exist as an option in the first place.

2. Positioning of the Row-Press Extension: The extension to handle Row-Press (MOAT-RP, Section 9, page 12) is valuable and demonstrates the framework's flexibility. However, its inclusion feels somewhat appended to the main narrative. Integrating this concept more smoothly, perhaps by framing the core problem as "data disturbance errors" with Rowhammer and Row-Press as two key instances, could create a more unified story and highlight MOAT's generalizability.

Questions to Address In Rebuttal

1. The authors present an attack on a FIFO-based Panopticon but discuss a "Drain-All-Entries" alternative in Appendix B. Could you elaborate on why you believe your assumed baseline is the more likely or canonical interpretation of the original Panopticon design? Furthermore, does the core insight of the Jailbreak attack—exploiting the time between a row's insertion into a queue and its eventual mitigation—apply to other queuing disciplines besides FIFO?

2. The proposed MOAT design cleverly uses a single-entry tracker (CTA) to minimize overhead. What are the fundamental trade-offs in moving to a multi-entry tracker (as explored for higher ABO levels in Section 8)? Does the security proof become significantly more complex, or are there other subtle vulnerabilities that emerge beyond the increased slowdown from longer stall times?

3. The analysis of the Ratchet attack (Section 5) is very insightful and assumes an attacker can precisely schedule activations between ALERTs. How sensitive is this attack's success to noise or scheduling jitter from the OS or other applications in a real system? Does this environmental noise provide any implicit, albeit unreliable, mitigation that might affect the calculated "Safe-TRH"?

Review Persona: Innovator

Summary

This paper proposes MOAT, a control logic and microarchitecture for implementing the JEDEC Per-Row Activation Counting (PRAC) and ALERT-Back-Off (ABO) framework for Rowhammer mitigation. The core of the proposed defense is a single-entry tracker per bank that stores the row with the highest activation count, governed by two thresholds: an Eligibility Threshold (ETH) for proactive mitigation and an ALERT Threshold (ATH) for reactive mitigation. The paper's other contributions are primarily in security analysis, where it introduces three novel attack patterns: 1) "Jailbreak," which breaks the FIFO queue design of the prior art Panopticon, 2) "Ratchet," which exploits the timing specifications of JEDEC ABO to exceed the nominal activation threshold, and 3) "Torrent-of-Staggered-ALERT" (TSA), a performance-degradation attack.

My evaluation focuses exclusively on the novelty of these contributions relative to existing prior art.

Strengths (Novelty-centric)

1. Novel Security Analysis of Prior Art: The "Jailbreak" attack (Section 3, page 5) is a genuinely novel contribution. It identifies a concrete and previously undocumented vulnerability in the Panopticon proposal [3], which is the direct inspiration for the JEDEC PRAC+ABO framework. The attack's insight—that a FIFO queue without stored counter values can be manipulated to delay mitigation for the youngest entry—is a specific and clever exploit. This finding alone is a valuable contribution to the field.

2. Novel Security Analysis of a New Standard: The "Ratchet" attack (Section 5.2, page 7) is another strong, novel contribution. It does not target a prior academic paper but rather the very recent JEDEC ABO specification itself. By demonstrating how an attacker can leverage the standard-defined allowance for a few activations between consecutive ALERTs, the authors uncover a fundamental limitation of the ABO mechanism. This analysis provides a novel upper bound on the security that any PRAC+ABO system can provide, which is an important finding for hardware designers.

3. Novel Architectural Simplification: While the foundational concepts of per-row counters and reactive alerts are not new (see Weaknesses), the specific architectural proposal of MOAT is a novel simplification over its direct predecessor, Panopticon. The decision to replace Panopticon's 8-entry queue with a single-entry "current-highest-aggressor" tracker (the CTA register, Figure 6, page 6) is a non-obvious design choice. It is justified by the novel insight that since proactive mitigation (during REF) can only service one aggressor at a time, a deep queue is not only unnecessary but, as shown by the Jailbreak attack, a liability. This "less is more" approach represents a novel design point in the PRAC+ABO implementation space.

Weaknesses (Novelty-centric)

1. Core Concepts are Not Novel: The paper builds upon a foundation of well-established prior art, which it correctly acknowledges. It is critical to distinguish the paper's novel control logic from the underlying concepts, which are not new.

   • Per-Row Counters: The idea of embedding activation counters within the DRAM array was disclosed in a 2012 patent filing [8] and later academically explored in Panopticon [3]. MOAT is an implementation of this idea, not its originator.
   • Reactive Signaling (ALERT): The concept of the DRAM signaling back to the memory controller to pause activity for mitigation was also central to the Panopticon proposal [3]. MOAT utilizes the standardized version (ABO) of this existing idea.

2. Control Principles are Refinements, Not Revolutions:

   • Dual Thresholds: The use of two thresholds (ETH and ATH) is a novel control logic in this specific context. However, multi-level thresholding is a common engineering pattern for resource management and is not a fundamentally new computer science concept. The novelty is in the refinement and application, not the invention of the principle.
   • "Track-the-Max" Principle: The TRR-Ideal concept from ProTRR [27] proposed mitigating the row with the highest activation count. MOAT’s CTA register effectively implements a practical version of this principle. The novelty of MOAT’s defensive mechanism, therefore, lies primarily in the synergistic combination of this principle with the ETH filter and the reactive ATH trigger within the JEDEC PRAC framework.

3. TSA Attack Pattern: The Torrent-of-Staggered-ALERT (TSA) attack (Section 7.3, page 10) is a new construction. However, the underlying principle—staggering accesses across different banks to create a sustained bottleneck—is a known technique in performance attacks and analysis. The novelty is limited to the specific application of this technique to the ALERT mechanism.

Questions to Address In Rebuttal

1. The paper’s core defensive innovation over prior art like Panopticon [3] and the TRR-Ideal concept [27] appears to be the combination of a single-entry tracker with the ETH/ATH dual-threshold logic. Could the authors clarify if this specific control pattern (a low threshold for eligibility/filtering and a high threshold for a hard stop/alert) has been proposed in other hardware security trackers, even outside the Rowhammer domain? This would help circumscribe the precise boundaries of the architectural novelty.

2. In Section 10.1, the authors compare MOAT to the concurrent work of Canpolat et al. [4], noting that the latter assumes an idealized "lookup of all DRAM rows" to find the maximum counter. While MOAT is clearly more practical, did this concurrent work also propose a control logic (e.g., thresholds) for managing mitigation, or was its novelty purely in the performance analysis of an idealized oracle?

3. The single-entry CTA tracker is justified based on the single-mitigation-per-REF-period limitation of proactive refresh. However, the reactive ALERT mechanism (especially at ABO levels 2 and 4) can mitigate multiple rows. Does the single-entry tracker design present any fundamental limitations in efficiently selecting candidates for a multi-row reactive mitigation, and is the proposed generalization in Section 8 (an L-entry tracker) the only viable approach?

Reviewer: The Guardian

Summary

This paper presents MoE-Lightning, a system designed for high-throughput batch inference of Mixture of Experts (MoE) models on memory-constrained GPUs. The core contributions are twofold: 1) CGOPIPE, a pipelining schedule that aims to finely overlap CPU computation, GPU computation, and I/O transfers (weights, KV cache) to maximize resource utilization, and 2) a Hierarchical Roofline Model (HRM) used as a performance model to guide the search for optimal inference policies (e.g., batch sizes, device placement for computations). The authors claim significant throughput improvements, up to 10.3x over existing systems like FlexGen, on low-cost hardware like a single T4 GPU.

Strengths

1. Problem Relevance: The paper addresses a critical and timely problem: deploying extremely large MoE models on commodity, memory-constrained hardware. This is a significant barrier to the broader adoption of these powerful models, and work in this area is of high interest.
2. Systematic Approach: The authors' approach of first building a theoretical performance model (HRM) and then using it to inform the design of a practical scheduling pipeline (CGOPIPE) is methodologically sound. The HRM provides a principled way to reason about performance bottlenecks in a heterogeneous system.
3. Thorough Experimental Comparison: The evaluation is conducted against relevant and strong baselines (FlexGen, DeepSpeed-Zero). The inclusion of controlled variants like FlexGen(c) (with CPU attention) and MoE-Lightning(p) (with padding) demonstrates a commendable effort to enable fair comparisons under specific conditions.

Weaknesses

My primary concerns with this submission relate to the interpretation and presentation of results, the validation of the core performance model, and the justification for key design choices.

1. Exaggerated and Potentially Misleading Headline Claim: The abstract and introduction prominently feature an "up to 10.3x higher throughput" claim. However, a deeper analysis of the evaluation (Section 5, Page 9) reveals this number is derived from comparing the authors' system with all optimizations (including variable-length request batching) against a baseline (FlexGen) that is forced to use padding. This is not an apples-to-apples comparison of the core scheduling technology. The more direct, padded-to-padded comparison (MoE-Lightning(p)) yields a much lower, though still significant, 3.5x improvement. The headline claim overstates the contribution of the core pipeline technique by conflating it with the benefits of a different batching strategy.

2. Inappropriate Use of "Super-Linear Scaling": In Section 5.3 (Page 10), the authors claim their system demonstrates "super-linear scaling" when moving from 2xT4 to 4xT4 GPUs. This term is technically incorrect and misleading. Super-linear scaling implies that doubling the resources more than doubles the performance (i.e., efficiency increases with scale). The mechanism described here is that increased aggregate GPU memory allows for a larger batch size, which better amortizes fixed overheads and moves the system out of a bottlenecked regime. While this is a positive result, it is not super-linear scaling; it is simply overcoming a bottleneck that was present at a smaller scale. This mischaracterization of a key result undermines the rigor of the analysis.

3. Insufficient Justification for the CPU Attention Design Choice: The decision to perform attention on the CPU is central to the CGOPIPE schedule. This is justified theoretically by the low operational intensity of attention (Figure 4, Page 5) and empirically in the ablation study (Figure 9, Page 11). However, the analysis in Figure 9 is incomplete. It compares the latency of the authors' CPU attention kernel against the latency of a KV cache transfer from CPU to GPU. It critically omits the baseline that matters most: the latency of an on-GPU attention kernel if the KV cache were already resident in GPU memory. The presented evidence only shows that their CPU attention is better than FlexGen's method of offloading (transferring KV cache), not that CPU attention is inherently better than GPU attention in an ideal scenario. This makes it difficult to assess whether CGOPIPE is making the best trade-off or simply a better trade-off than the baseline.

4. Lack of Empirical Validation for the HRM Performance Model: The HRM is presented as a foundational component for finding optimal policies. However, the paper provides no direct evidence validating the predictive accuracy of this model. Figure 10 (Page 12), which shows policy changes, is a product of the model's predictions, not an empirical validation of it. For the HRM to be a convincing contribution, the authors must demonstrate how well its latency/throughput predictions correlate with measured, real-world performance across a range of different (including suboptimal) policies. Without this validation, the HRM remains a theoretical construct of unproven utility.

5. Uncertain Generalizability: The entire system and its performance benefits appear to be highly tuned to a specific hardware regime: low-end GPUs (T4/L4) with relatively powerful host CPUs and a specific CPU-GPU interconnect bandwidth. The core finding that CPU attention is advantageous is highly sensitive to the relative performance of these components. It is unclear how these design choices and their benefits would translate to hardware with different characteristics, such as a high-end GPU (e.g., H100) paired with a proportionally less powerful CPU, where the balance of computation would be drastically different.

Questions to Address In Rebuttal

1. Please justify the use of the 10.3x throughput figure in the abstract and introduction. Given that this arises from comparing your un-padded system to a padded baseline, can you provide a more nuanced claim that clearly separates the gains from the CGOPIPE scheduler versus the gains from dynamic batching?

2. Regarding the claim of "super-linear scaling" (Section 5.3), please defend this terminology. Could you provide evidence that the performance-per-GPU increases with scale, or concede that a more accurate description would be "overcoming system bottlenecks with increased aggregate resources"?

3. In the analysis supporting CPU attention (Figure 9), could you provide the missing data point: the latency of a pure on-GPU attention implementation for the same micro-batch sizes on the L4 GPU? This is essential for understanding the true cost of offloading versus performing the computation on the GPU.

4. What steps were taken to validate the predictive accuracy of the Hierarchical Roofline Model (HRM)? Can you provide data, such as a parity plot, showing the correlation between HRM-predicted performance and empirically measured performance for a set of diverse inference policies?

5. How sensitive is the core design decision of using CPU attention to the underlying hardware? Could you use your HRM to model a scenario with a high-end GPU (e.g., an A100 or H100) and show whether the policy of offloading attention to the CPU still holds, or at what point the bottleneck shifts back to the GPU?

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper addresses the critical and timely problem of deploying large Mixture of Experts (MoE) language models on commodity, memory-constrained GPUs. The authors identify that while MoE models are computationally efficient for their size, their massive parameter count creates a significant memory bottleneck, making them inaccessible for many users. The core contribution is a co-designed system, MoE-Lightning, that combines a novel, fine-grained pipeline schedule (CGOPIPE) with a principled performance model (HRM) based on a Hierarchical Roofline Model. CGOPIPE meticulously orchestrates CPU computation, GPU computation, and multiple I/O data streams (weights, activations) to maximize resource utilization and hide data transfer latencies. HRM provides the analytical foundation to navigate the complex trade-offs and automatically find optimal scheduling policies (e.g., batch sizes, offloading ratios). The authors demonstrate impressive results, achieving up to a 10.3x throughput improvement for Mixtral 8x7B on a single T4 GPU compared to state-of-the-art offloading systems, and even show super-linear scaling when using tensor parallelism across multiple GPUs.

Strengths

1. Tackles a High-Impact, Practical Problem: The central thesis—making powerful but memory-hungry MoE models usable on affordable hardware—is of immense value to the research community and industry. As open-source models continue to grow, particularly with the MoE architecture, solutions that "democratize" access to them are not just useful but essential. This work sits squarely at the intersection of systems and machine learning, addressing a bottleneck that prevents widespread adoption of SOTA models.

2. Principled, Model-Driven System Design: The standout feature of this work is its analytical rigor. Instead of relying on purely empirical heuristics, the authors ground their system in a Hierarchical Roofline Model (HRM) (Section 3, pg. 3-5). This extension of the classic Roofline model to a heterogeneous system with multiple memory tiers (CPU DRAM, GPU HBM) and compute units is an elegant way to reason about performance bottlenecks. It provides a clear, visual language for understanding when a workload is bound by PCIe bandwidth, GPU compute, or CPU memory bandwidth. This model-driven approach is a significant strength, allowing the system to find optimal configurations rather than relying on manual tuning.

3. Sophisticated Pipeline Scheduling (CGOPIPE): The proposed CGOPIPE schedule (Section 4.1, pg. 6) is the technical heart of the paper and a clear advance over existing offloading techniques. Figure 6 on page 7 provides an excellent visualization of its efficiency. While systems like FlexGen also pipeline execution, CGOPIPE’s fine-grained interleaving of paged weight transfers, CPU-based attention computation, and GPU-based FFN execution appears to minimize idle "bubbles" much more effectively. The decision to perform attention on the CPU, informed by the HRM analysis, is a key insight that frees up crucial I/O bandwidth for transferring the much larger expert weights.

4. Exceptional Empirical Results: The performance gains reported are not merely incremental; they represent a step-function improvement in what is achievable on low-end hardware. The end-to-end throughput results (Figure 7, pg. 9), especially the 10.3x speedup on a T4 GPU, are highly compelling. Furthermore, the demonstration of super-linear scaling with tensor parallelism (Section 5.3, pg. 10) is a powerful result. It suggests that prior systems were so fundamentally limited by a bottleneck (likely I/O or CPU memory) that simply adding more GPU memory capacity and bandwidth unlocks disproportionately large performance gains, a bottleneck that MoE-Lightning effectively mitigates.

Weaknesses

From a synthesizer's perspective, the weaknesses are less about flaws in the work itself and more about its positioning and the boundaries of its contribution.

1. Contextualization Beyond MoE Models: The paper is heavily framed around the unique properties of MoE models (very high memory-to-compute ratio). While the authors briefly mention in the further discussion that the techniques are applicable to dense models (Section B.1, pg. 13), the work would be stronger if it provided more context here. For a dense model like Llama 2 70B, which also requires offloading on a T4, how would the bottlenecks identified by HRM shift? One might surmise that attention (and its KV cache) becomes a more dominant factor relative to weight loading. A brief analysis of this would help contextualize MoE-Lightning as a general solution for memory-constrained inference, rather than just an MoE-specific one.

2. Exclusive Focus on Throughput-Oriented Workloads: The entire evaluation is centered on maximizing throughput for offline, batch-processing workloads. This is a valid and important use case (e.g., data processing, summarization). However, a significant portion of LLM deployment is for interactive, latency-sensitive services. The paper does not discuss how the CGOPIPE scheduling and large batch sizes would perform in a low-latency, single-user (or small batch) scenario. While this is a limitation of scope rather than a flaw, acknowledging this trade-off more explicitly would help readers understand the ideal application domain for this system.

3. The Broader Landscape of Co-Design: This work is an excellent example of hardware-software co-design, where the system is optimized for a specific hardware reality (slow PCIe, fast GPU compute, etc.). It fits into a broader trend of heterogeneous computing for LLMs seen in works like FastDecode, PowerInfer, and others. The paper could benefit from a slightly expanded discussion in the Related Work section (Section 7, pg. 12) to better situate itself within this landscape, highlighting how its focus on model-driven scheduling for both weights and activations under extreme memory pressure differentiates it from systems that might focus more on activation sparsity or different CPU/GPU task divisions.

Questions to Address In Rebuttal

1. Regarding the HRM performance model (Section 4.2, pg. 8), the paper mentions it uses theoretical flops/bytes combined with profiled hardware peaks to guide the policy search. How sensitive is the final throughput to the accuracy of this model? Have you validated that the policy chosen by HRM is indeed close to the empirically-determined optimal policy? A brief analysis of the model's predictive power would strengthen the claim of a "principled approach."

2. The claim of "super-linear scaling" is very strong and interesting. Could you elaborate on the underlying system dynamics that enable this? My hypothesis is that with 2xT4s, the system is still fundamentally constrained (e.g., by CPU memory or a batch size limit), while the 4xT4 configuration provides enough aggregate memory to cross a threshold, allowing for a batch size that fundamentally changes the operational intensity to better saturate the GPUs. Is this interpretation correct, or is there another mechanism at play?

3. Could the authors comment on the applicability of the CGOPIPE pipeline for latency-critical scenarios? If you were to optimize for first-token latency or time-per-output-token for a single user (batch size = 1), would the strategy of offloading weights layer-by-layer and performing attention on the CPU still be optimal? What does the HRM predict would be the primary bottleneck in such a setting?

Reviewer: The Innovator (Novelty Specialist)

---

Summary

This paper presents MoE-Lightning, a system for high-throughput inference of Mixture-of-Experts (MoE) models on GPUs with limited memory. The central problem is well-established: MoE models are memory-intensive, and offloading to CPU memory is necessary on consumer hardware, creating an I/O bottleneck. The authors propose two primary contributions to address this: (1) CGOPIPE, a novel CPU-GPU-I/O pipeline schedule that aims to maximize resource utilization by overlapping CPU-based attention computation with GPU-based FFN computation and fine-grained data transfers; and (2) HRM, a Hierarchical Roofline Model designed to analyze performance bottlenecks in this heterogeneous setting and guide the search for optimal inference policies. The paper demonstrates significant throughput improvements over existing systems like FlexGen and DeepSpeed-Zero.

My review focuses exclusively on the novelty of these two core contributions.

---

Strengths

1. Novelty in Synthesis and Orchestration (CGOPIPE): The primary novel contribution of this work lies in the specific design of the CGOPIPE schedule (Section 4.1, page 6). While the constituent ideas—offloading computation to the CPU, overlapping I/O with computation, and even using the CPU for attention—have been explored in prior work, the authors' synthesis is non-trivial. The key novelty is the fine-grained orchestration that interleaves the transfer of multiple data types (paged weights for layer i+1, hidden states for micro-batch j+1, and QKV values for micro-batch j+2) to minimize pipeline bubbles. As visualized in Figure 6 (page 7), this schedule is more intricate than the coarser-grained prefetching in FlexGen [42] or the CPU-attention pipeline in FastDecode [17] (which does not consider weight offloading). This specific orchestration for the MoE offloading scenario appears to be novel.

2. Novel Application and Extension of a Modeling Framework (HRM): The HRM (Section 3.2, page 4) is presented as a novel extension to the classic Roofline Model [48]. The concept of extending Roofline is not new; however, the authors' specific formulation for the CPU-GPU offloading problem is a pragmatic and useful contribution. The introduction of a "Memory Roof from level j to i" (Eq. 6), which explicitly models the performance limitation imposed by the CPU-GPU interconnect (e.g., PCIe bandwidth), is a clean and effective way to reason about the trade-offs of offloading. The subsequent identification of new "turning points" (Eqs. 9 and 10) provides a principled method for deciding whether a given operation is bound by the interconnect, local memory bandwidth, or compute, which is a novel application of this modeling style to the LLM offloading problem.

---

Weaknesses

1. Incremental Nature of Contributions: The core weakness, from a novelty standpoint, is that the paper's contributions are more integrative than foundational. The work excels at cleverly combining and refining existing concepts rather than inventing fundamentally new ones.

   • CGOPIPE: The building blocks are known. FlexGen [42] established the layer-by-layer offloading and I/O-compute overlap paradigm. FastDecode [17] proposed overlapping CPU attention with GPU computation. The concept of "paging" is heavily inspired by PagedAttention from vLLM [26], though applied here to weights. The novelty is therefore confined to the specifics of the schedule, which, while effective, represents an advanced engineering optimization of known principles.
   • HRM: The idea of extending the Roofline model to account for multiple memory levels or heterogeneous processors is not new in the high-performance computing literature. The paper does not position its HRM against these prior Roofline extensions, making the scope of its novelty appear larger than it may be. The contribution is better described as a domain-specific adaptation of the Roofline methodology rather than a new modeling paradigm.

2. Insufficient Disambiguation from Prior Art: The paper could do a better job of precisely delineating its novel "delta" from the closest prior work.

   • In the discussion of CGOPIPE, the distinction from FlexGen's prefetching mechanism is not made explicit. The key difference appears to be the fine-grained, paged nature of the weight transfer, which allows for better interleaving, but this is not clearly articulated as the central point of novelty.
   • The term "Hierarchical Roofline Model" is introduced without sufficient context of other hierarchical or multi-level Roofline models in the literature. This makes it difficult to assess the exact conceptual leap being made.

---

Questions to Address In Rebuttal

1. Regarding CGOPIPE: Can the authors precisely articulate the core algorithmic difference between CGOPIPE's scheduling and the prefetching mechanisms in FlexGen [42]? Is the key innovation the "paging" of weights to enable finer-grained transfer interleaving, as opposed to a monolithic layer prefetch? A clearer statement on this would strengthen the novelty claim.

2. Regarding HRM: Please clarify the novelty of HRM in the context of prior work that has also extended the Roofline model to account for memory hierarchies or heterogeneous processors. What is the precise delta between HRM and these existing extensions? Providing citations and a brief comparison would help situate the contribution accurately.

3. Regarding Complexity vs. Benefit: The proposed CGOPIPE schedule introduces significant scheduling complexity. The HRM model, on the other hand, relies on theoretical peak performance values. How does the real-world performance of the policy found by HRM compare to a policy found through a brute-force search over a small, discrete set of parameters? This would help clarify whether the novel modeling framework is essential for achieving the reported performance, or if the gains are primarily from the novel pipeline structure itself.

Paper Title: MVQ: Towards Efficient DNN Compression and Acceleration with Masked Vector Quantization Reviewer: The Guardian (Adversarial Skeptic)

---

Summary

The paper proposes MVQ, a DNN compression scheme that sequentially combines N:M pruning with vector quantization (VQ). The core algorithmic contribution is a "masked k-means" algorithm that performs clustering only on the unpruned weights within a vector, aiming to reduce the clustering error for important weights. At the architectural level, the authors propose a modified EWS-dataflow accelerator featuring a sparsity-aware systolic array designed to exploit the structure of MVQ. The authors claim significant improvements in model accuracy over other VQ methods at similar compression ratios, and substantial hardware benefits, including a 2.3x boost in energy efficiency and a 55% reduction in systolic array area compared to a baseline EWS accelerator.

While the proposed algorithm appears logically sound and demonstrates strong empirical performance against other VQ techniques, the hardware evaluation contains significant methodological weaknesses and overstated claims that undermine the central conclusions about accelerator efficiency. The comparisons to both internal baselines and external state-of-the-art accelerators are not conducted on a level playing field, making it difficult to ascertain the true architectural contribution of this work.

Strengths

1. Sound Algorithmic Premise: The core motivation—that forcing important weights to be clustered with zero-valued pruned weights is detrimental—is valid. The proposed masked k-means algorithm is a direct and logical solution to this problem.
2. Strong Algorithmic Evaluation: The ablation study in Section 6.3 (Table 3, page 9) effectively demonstrates that masked k-means (Case D) significantly reduces clustering error for important weights and improves accuracy compared to naively applying k-means to a sparse weight tensor (Case C).
3. Favorable Comparison to VQ Methods: The paper shows consistently better accuracy and lower Sum of Squared Errors (SSE) compared to other VQ-based methods like PQF and BGD (Figure 13 and Table 5, page 10) at similar compression ratios. This suggests the algorithmic component of the work is a genuine improvement.

Weaknesses

1. Fundamentally Flawed Comparison to State-of-the-Art Accelerators: The comparison against prior sparse accelerators in Table 9 (page 13) is misleading. The authors claim a 1.73x higher energy efficiency over the prior art, specifically highlighting a 73% improvement over S2TA. However, this comparison is invalid as the workloads are different. The MVQ accelerator is evaluated on ResNet18, while S2TA is evaluated on AlexNet. ResNet-style architectures exhibit significantly higher data reuse and operational intensity compared to AlexNet, making them inherently more efficient to accelerate on systolic arrays. Any reported efficiency gain is therefore a convolution of architectural improvements and a more favorable workload. This comparison does not constitute a fair, scientific benchmark.

2. Overstated and Misleading Hardware Claims: The abstract and conclusion claim a "55% reduction in the size of the systolic array." While technically true for the array itself in the 64x64 configuration (calculation is closer to 50%: (4.236-2.129)/4.236 = 49.7% from Table 7), this is a classic "cherry-picking" of data. The systolic array is only one component of the chip. The total accelerator area, including L1/L2 caches and other components, is not reduced by nearly this much. This selective reporting inflates the perceived benefit of the proposed architecture.

3. Lack of Justification for Pruning Strategy: The paper adopts N:M pruning but provides little justification for why this specific structured sparsity pattern is optimal when paired with VQ. The pruning strategy experiments in Section 6.2 (page 8) only explore different N:M ratios and layerwise vs. cross-layer application, but do not compare against other structured or even unstructured pruning methods that might synergize differently with the subsequent VQ step. The choice of N:M seems driven more by its hardware friendliness than by a rigorous analysis of its interaction with VQ.

4. Inconsistent Evaluation Workloads: The algorithm is validated on a broad set of tasks, including image classification, object detection (MaskRCNN), and segmentation (DeepLab-v3) in Section 6. However, the entire hardware evaluation in Section 7 is performed only on classification models (ResNet, VGG, AlexNet, MobileNet). Models like MaskRCNN have vastly different layer dynamics and memory access patterns. There is no evidence provided that the reported hardware gains (e.g., data access reduction in Figure 15) would hold for these more complex, non-classification workloads.

5. Ambiguity in Compression Ratio Fairness: The paper's headline claim is improved accuracy at comparable compression ratios. However, the MVQ method introduces a storage overhead for the mask indices (bm in Equation 7). When comparing to methods like PQF at "~22x compression", it is unclear if this overhead was properly accounted for. A fair comparison would grant the baseline method a slightly larger codebook budget to equate the total storage cost (codebook + indices) of both methods, not just the nominal compression ratio. The authors do not specify if this was done.

Questions to Address In Rebuttal

1. Please provide a justification for comparing the proposed accelerator's performance on ResNet18 against S2TA's performance on AlexNet in Table 9. To make a fair claim of superiority, could the authors provide performance numbers for their accelerator running AlexNet, or re-implement S2TA's core principles and evaluate it on ResNet18?

2. Regarding the claimed 55% area reduction: please clarify the percentage reduction in total chip area (or at least the total accelerator subsystem area including L1/L2/Controllers), not just the systolic array. This would provide a more honest representation of the area savings.

3. In the comparisons against PQF (Table 5), how was the compression ratio of ~22x for the baseline determined? Was the storage overhead of the MVQ mask (bm) accounted for by giving the PQF baseline a slightly larger codebook budget to ensure the total model size was identical? If not, the comparison is not truly at an equal compression ratio.

4. Given the significant architectural differences between classification models and models like MaskRCNN, can the authors provide any hardware performance data (e.g., energy efficiency, speedup) for at least one non-classification model to substantiate that the architectural benefits are general and not confined to CNNs for classification?

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper presents MVQ, a novel algorithm-hardware co-designed framework for deep neural network compression and acceleration. The work identifies a key limitation in conventional vector quantization (VQ): its inability to differentiate between important and unimportant weights within a sub-vector, leading to suboptimal codebooks and accuracy degradation.

The core algorithmic contribution is a two-stage process. First, fine-grained structured N:M pruning is applied to remove less salient weights. Second, a novel "masked k-means" algorithm is used to generate a VQ codebook, where the clustering objective function explicitly ignores the pruned weights. This ensures that the codebook's limited representational capacity is focused exclusively on approximating the remaining, important weights.

On the hardware side, the authors propose a custom accelerator based on the Enhanced Weight Stationary (EWS) dataflow. This architecture features a sparsity-aware systolic array specifically designed to exploit the N:M sparsity created by MVQ, skipping zero-valued computations within each processing element (PE) group to save power and reduce computational resources. The result is a synergistic system where the compression algorithm's properties are directly mapped to an efficient hardware implementation. The authors validate their approach across a range of models and tasks, demonstrating superior accuracy at high compression ratios and significant improvements in hardware energy efficiency compared to baseline and prior sparse accelerators.

Strengths

1. Elegant and Intuitive Core Idea: The fundamental premise of this work is exceptionally strong. The insight that conventional VQ "wastes" its representational power on zero- or near-zero-valued weights is a crucial one. The proposed solution—using a mask to focus the k-means clustering process—is a direct, elegant, and principled way to solve this problem. The empirical observation presented in Section 4.1 (page 3, Figure 1) provides a clear and compelling motivation for the entire approach.

2. Excellent Algorithm-Hardware Co-Design: This paper is a prime example of successful co-design. The algorithmic choice of N:M pruning is not arbitrary; it creates a regular sparsity pattern that is amenable to hardware acceleration. The proposed "Sparsity-aware Systolic Array" (Section 5.3, page 7) is not a generic sparse accelerator but is tailored to exploit the specific structure of MVQ, using cascaded Leading Zero Counters (LZCs) to efficiently skip computations. This tight coupling between the algorithm's output structure and the hardware's capabilities is the paper's greatest strength and leads to the impressive efficiency gains reported.

3. Contextualization and Strong Results: The work is well-positioned within the existing literature. It correctly identifies the limitations of prior VQ methods (e.g., PQF, BGD) and provides a direct comparison. The reported results are significant. Achieving a 1.73x higher energy efficiency over prior state-of-the-art sparse accelerators (Table 9, page 13) is a substantial improvement. Furthermore, demonstrating that this method not only compresses the model but also significantly reduces FLOPs (Table 3, page 9) highlights its dual benefit for both storage and computation.

4. Broad and Thorough Evaluation: The authors have validated their method comprehensively. The evaluation spans multiple application domains (classification, object detection, segmentation), a diverse set of network architectures (from legacy VGG/AlexNet to modern ResNets and MobileNets), and a detailed hardware analysis (area, power, performance scaling). This breadth gives confidence in the generalizability and robustness of the proposed MVQ framework.

Weaknesses

1. Positioning Relative to Foundational Work: While the paper does an excellent job comparing against contemporary VQ-based methods, it could benefit from more explicitly distinguishing its approach from the classic "Deep Compression" pipeline (Han et al., 2015). Deep Compression also combines pruning and quantization. The key philosophical difference is that MVQ integrates the pruning mask into the clustering objective itself, whereas the classic pipeline treats them as more separate, sequential steps. Highlighting this conceptual advance more directly would further strengthen the paper's claimed novelty.

2. Practical Training Complexity: The overall compression pipeline illustrated in Figure 2 (page 4) appears to involve several distinct stages: initial grouping, pruning and fine-tuning, masked k-means clustering, and final codebook fine-tuning. This multi-stage process, while effective, may introduce significant complexity and increase the total training time required to compress a model. A brief discussion of the practical overheads of this pipeline would provide a more complete picture for potential adopters.

3. Hardware Scalability Concerns: The hardware design for the "Parallel Masked CodeBook RF Read Out" (Figure 6, page 6) requires L/d read ports on the Codebook RF to service a systolic array row of width L. While feasible for the tested configurations, this could present a scalability challenge for very wide arrays or for VQ with small sub-vector dimensions (d), potentially leading to significant area and routing congestion for the codebook register file.

Questions to Address In Rebuttal

1. Could the authors elaborate on the key conceptual difference between MVQ's integrated pruning-clustering approach and the sequential pruning-then-quantization pipeline used in seminal works like Deep Compression?

2. Can the authors provide insight into the training overhead of the proposed four-step MVQ pipeline? For instance, how does the end-to-end time to produce a compressed model compare to a standard VQ fine-tuning process?

3. Regarding the hardware architecture, have the authors considered the scalability of the multi-ported Codebook RF? How would the design and its area/power costs be affected in a much wider systolic array (e.g., 256x256) or when using a smaller VQ block size (d=4)?

Reviewer: The Innovator (Novelty Specialist)

---

Summary

This paper proposes "Masked Vector Quantization" (MVQ), a method for DNN compression that combines N:M structured pruning with vector quantization (VQ). The authors identify that conventional VQ degrades accuracy by failing to preserve important weights. Their proposed solution is a two-stage process: first, prune less important weights using an N:M pattern, and second, apply a novel "masked k-means" algorithm for VQ. The core idea of this algorithm is to perform the k-means clustering steps (distance calculation for assignment and centroid updates) by exclusively considering the unpruned weights, effectively ignoring the pruned weights during codebook generation. This algorithmic novelty is paired with a co-designed hardware accelerator based on the EWS dataflow, featuring a sparsity-aware systolic array that skips computations for the pruned weights to improve efficiency.

Strengths

The paper's primary strength lies in its clearly defined and well-motivated algorithmic novelty.

1. Novel Formulation of the VQ Objective: The core novel contribution is the "masked k-means" algorithm (Section 4.4, page 4). While combining pruning and quantization is not new, the authors' approach of integrating a binary mask directly into the k-means objective function (Equations 1-3, page 5) is a distinct and clever idea. Prior art typically treats pruning and VQ as sequential, decoupled steps (prune, then cluster the resulting sparse tensor). By masking the distance metric and the centroid update rule, the authors ensure the codebook is optimized only for representing the important, unpruned weights, preventing the numerous zero-valued pruned weights from corrupting the centroids. This is a conceptually clean and significant departure from conventional VQ application in the DNN compression space.

2. Novel Synthesis of Architectural Concepts: While the individual architectural components are not entirely new (EWS dataflow is from prior work [35], N:M sparsity acceleration has been explored in [21]), their synthesis to support the MVQ algorithm is novel. The design of an "assignment-aware weight loader" (Section 5.2, page 7) that reconstructs sparse weight vectors on-the-fly from a codebook, index, and a compressed mask representation is a specific solution tailored to the MVQ algorithm. The integration of this decompression logic with a sparsity-aware systolic array built upon the EWS dataflow represents a novel co-design effort.

Weaknesses

My critique focuses on situating the novelty within the broader context of prior art and the justification for the increased complexity.

1. Limited Acknowledgment of Broader Prior Art in Clustering: The concept of performing clustering on incomplete or "masked" data is a well-established subfield in machine learning and statistics, often referred to as "clustering with missing values." The fundamental idea of computing distances and means using only available features is not new in that context. The paper presents "masked k-means" as a wholly new concept without acknowledging this extensive prior art. The novelty here is therefore not the invention of masked clustering, but its specific application and formulation for the DNN weight compression problem, where "missing" values are intentionally created via magnitude pruning. The paper would be stronger if it framed its contribution more precisely in this light.

2. Architectural Novelty is Primarily Synthetic: The paper's architectural contribution is the novel integration of known techniques rather than the invention of new ones. The use of Leading Zero Counters (LZCs) to encode sparsity and enable compute-skipping (Figure 8, page 7) is a common pattern in sparse accelerator design. Similarly, systolic arrays for N:M sparsity have been proposed, for example in S2TA [21]. The authors should be more explicit that their architectural novelty lies in the specific co-design choices required to fuse a VQ decompression pipeline with an N:M sparse EWS dataflow, rather than implying the foundational techniques are new.

3. Incremental Gain for Added Complexity: The central premise is that MVQ better preserves important weights, leading to higher accuracy. The experimental results, while positive, show a somewhat marginal improvement over the closest prior art. For example, on ResNet-50 (Figure 13, page 10), MVQ achieves 75.2% accuracy, a 1.0% improvement over PQF [23] at a ~22x compression ratio. While this is coupled with a significant FLOPs reduction (due to pruning), the algorithmic complexity has increased (requiring mask storage and a more complex clustering process). The trade-off between this added complexity and the resulting accuracy gain could be viewed as incremental rather than transformative.

Questions to Address In Rebuttal

1. Could the authors please contrast their "masked k-means" algorithm with established methods for k-means on data with missing values? Please clarify how your formulation for structured pruning-induced sparsity is distinct from these more general approaches and why they would be unsuitable for this task.

2. The sparse systolic array tile in Figure 8 shows a design using cascaded LZCs to handle N:M sparsity. Could you provide a more detailed comparison to the mechanisms used in prior N:M accelerators like S2TA [21]? What are the specific trade-offs (e.g., area, latency, control complexity) of your approach versus others, and why is your design particularly well-suited for a VQ-based model running on an EWS dataflow?

3. The paper argues that approximating important weights is key. Have you explored alternatives to N:M pruning for generating the mask? For instance, would an unstructured mask (albeit with higher metadata cost) allow the masked k-means algorithm to generate an even better codebook, potentially pointing to the upper bound of the proposed algorithm's effectiveness? This would help isolate the novelty of the masked clustering from the choice of pruning scheme.

Reviewer Persona: The Guardian (Adversarial Skeptic)

---

Summary

The authors present GPULOG, a Datalog engine designed for execution on GPUs. The central contribution is a novel data structure, the Hash-Indexed Sorted Array (HISA), which aims to combine the benefits of efficient range-queries and parallel iteration suitable for the GPU's SIMT architecture. The paper claims that this system significantly outperforms a state-of-the-art CPU-based engine (Soufflé) by up to 45x on specific workloads, and also shows favorable performance and memory characteristics compared to other GPU-based join approaches. While the engineering effort is apparent and the HISA data structure is conceptually sound, the paper's central performance claims rest on a potentially inequitable experimental comparison, and several key design choices lack the rigorous quantitative justification required for a top-tier publication.

Strengths

1. Well-Motivated Problem: The paper correctly identifies a key bottleneck in modern, multi-threaded CPU-based Datalog engines: serialization and locking overhead in core data structures, particularly for deduplication and indexing (Section 1, Page 1). This provides a solid foundation for exploring alternative hardware architectures.

2. Sound Data Structure Design: The HISA data structure (Section 4, Page 4) is a clever design. Separating the dense data array (for parallel iteration) from a sorted index array (for range queries) and using a hash table solely as an entry point into the sorted ranges is an intelligent way to manage GPU memory and execution patterns. It correctly addresses the requirements laid out in Section 3.

3. Insightful Performance Analysis: The breakdown of execution time in Figure 6 (Page 11) is commendable. It honestly reveals that merging relations ("Merge Delta/Full") constitutes a major bottleneck (~40-50% of runtime), even more than the join operations themselves. This level of self-assessment is a sign of a thorough, if not complete, evaluation. The analysis attributing the CSPA speedup to memory bandwidth, supported by Soufflé's low CPU utilization and the microbenchmark in Table 6, is a strong piece of investigative work.

Weaknesses

1. Fundamentally Unbalanced Performance Comparison: The headline claim of a "45x gain" (Abstract, Page 1; Table 4, Page 11) is derived from comparing a top-of-the-line data center GPU (NVIDIA H100 with 3.35 TB/s HBM) against a multi-core CPU (AMD EPYC 7543P with ~200 GB/s DDR4). The authors themselves correctly identify the workload (CSPA) as memory-bound (Section 6.5, Page 10). Given this, the massive performance delta is less a testament to the novelty of GPULOG's logic and more an expected outcome of running a memory-bound problem on hardware with over an order of magnitude more memory bandwidth. This is an apples-to-oranges comparison that inflates the perceived contribution. For the claim to be robust, a comparison against a CPU system optimized for memory bandwidth would be necessary.

2. Insufficient Justification for Key Optimizations:

   • Temporarily-Materialized n-way Joins (Section 5.2, Page 7): This optimization is presented as a definitive improvement for handling workload imbalance within a GPU warp. However, this claim is supported only by a small, illustrative diagram (Figure 5) and lacks empirical validation. Materializing intermediate results and launching a separate kernel introduces significant overhead. There is no ablation study to quantify the trade-off. It is plausible that for many join patterns (e.g., those with high selectivity or small intermediate results), this approach would be substantially slower than a fused, non-materialized operator. The paper presents a specific tactic as a general solution without providing the evidence.
   • Eager Buffer Management (EBM) (Section 5.3, Page 8): The evaluation in Table 1 (Page 9) demonstrates that EBM improves runtime at the cost of increased peak memory usage (e.g., a 32% increase for usroads). The authors frame this as a successful optimization but fail to adequately discuss the trade-off. In memory-constrained scenarios, this "eager" allocation could be the difference between running and failing with an out-of-memory error. The tunability of the parameter k is mentioned but no sensitivity analysis is provided to guide its selection.

3. Understated Limitations of the HISA Data Structure:

   • Sorting Overhead: The construction of HISA requires a full sort of the index array based on the join columns (Section 4.2, Algorithm 1, Page 5). This sorting cost is non-trivial, especially in an iterative context where relations are constantly being merged. The performance breakdown in Figure 6 lumps this cost into "Indexing Full/Delta," obscuring the specific overhead of sorting. For workloads with a very large number of fixed-point iterations where only a few tuples are added each time, the cost of repeatedly sorting or merging into these sorted structures could dominate.
   • Hash Function Collisions: The hash table maps the hash value of the join columns to an index (Section 4.3, Page 5). The paper does not discuss the performance implications of hash function collisions (i.e., hash(key1) == hash(key2) where key1 != key2). While the subsequent linear scan of the sorted index array would still yield the correct result, a high collision rate would nullify the O(1) benefit of the hash table lookup, causing threads to perform long, unnecessary scans from an incorrect starting position. The robustness of the design against non-ideal key distributions is unproven.

Questions to Address In Rebuttal

1. Regarding the headline 45x speedup claim: Can you justify why the comparison between an H100 GPU and an EPYC 7543P CPU is technically fair for a memory-bound workload? To strengthen this claim, would you consider comparing against a CPU system with the highest available memory bandwidth (e.g., a server with 12-channel DDR5) to create a more architecturally equitable baseline?

2. For the "Temporarily-Materialized n-way Joins" optimization (Section 5.2): Please provide a rigorous ablation study. Show performance data with and without this optimization on several queries and datasets. What are the precise characteristics of a join (e.g., selectivity, intermediate relation size, variance in work per-thread) that determine whether this strategy is beneficial versus detrimental?

3. Regarding the HISA hash table (Section 4.3): Please clarify the exact mechanism and performance implications of hash function collisions (distinct join keys producing the same hash value). How does performance degrade as the collision rate increases? Have any tests been run on datasets with adversarial key distributions?

4. Can you provide a more detailed performance breakdown that isolates the cost of sorting within the "Indexing" phase shown in Figure 6? How does this specific sorting cost scale with the number of tuples being merged in each iteration?

Reviewer: The Synthesizer (Contextual Analyst)

Review Form

Summary

This paper presents GPULOG, a Datalog engine backend designed from the ground up for high-performance execution on modern GPUs. The core contribution is a holistic system design that pairs a novel data structure, the Hash-Indexed Sorted Array (HISA), with GPU-aware evaluation strategies to overcome the traditional bottlenecks of running iterative relational algebra on a massively parallel architecture. HISA is a three-tiered structure designed to provide efficient range-querying, parallel iteration, and deduplication—all critical for Datalog's semi-naïve evaluation. The authors augment this data structure with two key algorithmic optimizations: temporarily-materialized n-way joins to mitigate thread divergence, and eager buffer management to amortize the high cost of memory reallocation within the fixpoint loop.

The authors demonstrate the efficacy of their approach through a comprehensive evaluation against state-of-the-art CPU (Soufflé) and GPU (GPUJoin, cuDF) systems. The results are particularly compelling in the domain of program analysis, where GPULOG achieves a remarkable speedup of up to 45x over a highly-optimized, multi-core CPU engine on a context-sensitive points-to analysis of PostgreSQL. This work provides a strong blueprint for building high-performance declarative query engines on accelerator hardware.

Strengths

1. A Clear Problem Framing and an Elegant Solution: The authors do an excellent job in Section 3 (page 3) of distilling the challenges of GPU Datalog into four concrete requirements: [R1] range-querying, [R2] parallel iteration, [R3] multi-column joins, and [R4] deduplication. The proposed HISA data structure (Section 4, page 4) is not just a collection of optimizations but a principled solution that directly and elegantly addresses this set of conflicting demands. This clarity of thought, connecting the problem definition directly to the technical solution, is a major strength of the paper.

2. Demonstrated Impact on a Significant, Real-World Problem: The paper's most significant achievement is its application to Context-Sensitive Program Analysis (CSPA). This is not a toy problem; it is a memory-bandwidth-intensive workload that represents a major bottleneck in software engineering and security analysis. By showing a 35-45x performance improvement over Soufflé on large codebases like PostgreSQL (Table 4, page 11), the authors convincingly argue that their system can fundamentally change the performance envelope for this entire class of applications. This makes the work not just an academic exercise but a potentially transformative piece of engineering.

3. Holistic, Architecture-Aware System Design: The contribution here is more than just a clever data structure. It is the co-design of the data structure with the algorithms that operate on it. The identification of n-way join workload imbalance as a source of warp inactivity (Figure 5, page 8) and the subsequent solution of temporary materialization is a prime example of deep, architecture-specific thinking. Similarly, the breakdown in Figure 6 (page 11) correctly identifies the Merge Delta/Full phase as a major bottleneck, which provides clear motivation for the Eager Buffer Management strategy. This holistic approach is what separates a simple port from a truly optimized system.

Weaknesses

This is a strong paper, and the following points are primarily suggestions for strengthening the context and discussion rather than identifying fundamental flaws.

1. Positioning Relative to the Broader GPU Database Literature: While the authors correctly compare against GPUJoin and cuDF, the novelty of HISA could be situated more firmly within the broader academic landscape of GPU data structures for analytics. The individual components—sorted arrays for range scans and hash tables for point lookups—are well-established primitives. The key innovation is their specific three-tiered combination and the insight of hashing to an index in a sorted array. A brief discussion contrasting HISA with other hybrid structures (e.g., GPU-based B-trees or radix trees, as seen in other database work) would help to more sharply define its unique advantages for the iterative, join-heavy Datalog workload.

2. Exploring the Boundaries of the Approach: The paper demonstrates stellar performance on workloads that are emblematic of Datalog's strengths (transitive closure, program analysis). It would be beneficial to briefly discuss the potential limitations or performance characteristics of GPULOG on Datalog programs with different structures. For instance, how would it handle queries dominated by very wide relations (where tuple copying could be expensive) or those involving complex aggregations (which the authors note as future work)? This would add valuable nuance and help readers understand the scope of the current contributions.

Questions to Address In Rebuttal

1. The HISA data structure elegantly balances several performance concerns. Could you elaborate on the trade-offs, particularly regarding memory overhead and construction time? For example, how does the combined memory footprint of the data array, sorted index array, and hash table compare to a more naive representation (e.g., just a tuple array) or the on-disk size of the input relations?

2. The strategy of temporarily materializing intermediate results for n-way joins is clever for ensuring warp efficiency. However, it trades computation (and potential thread idleness) for increased memory traffic and a synchronization point between kernel launches. Could you comment on the performance envelope for this optimization? Is it always a net-win, or are there scenarios (e.g., joins with low selectivity) where a single, non-materialized kernel might be preferable?

3. The performance gains on memory-bandwidth-limited workloads like CSPA are outstanding. This suggests that the core ideas behind HISA and your GPU-aware evaluation strategy might be broadly applicable. Could you speculate on the applicability of this approach beyond Datalog to other systems that rely on iterative relational algebra, such as in graph machine learning frameworks (e.g., message-passing GNNs) or certain scientific computing simulations?

Reviewer: The Innovator (Novelty Specialist)

Summary

The authors present GPULOG, a Datalog engine designed for high-performance execution on GPUs. The paper's core thesis is that by using a specific set of data structures and evaluation strategies, significant speedups can be achieved over modern CPU-based engines like Soufflé. The work claims novelty in three primary areas: (1) a data structure called the Hash-Indexed Sorted Array (HISA); (2) the strategy of temporarily materializing intermediate results for n-way joins to improve GPU resource utilization; and (3) an Eager Buffer Management (EBM) technique to amortize memory reallocation costs. The paper demonstrates substantial performance gains on several benchmarks, particularly a context-sensitive points-to analysis.

Strengths

The primary strength of this work lies in the effective synthesis and meticulous engineering of several known computational principles into a cohesive system tailored for the specific problem of GPU-based Datalog evaluation. The authors have clearly identified key performance bottlenecks in the SIMT execution model—namely thread divergence and memory allocation overhead—and have applied targeted engineering solutions to mitigate them. The performance results, particularly the 45x gain over a highly-optimized CPU engine (Table 4, Page 11), are impressive and indicate that the authors' implementation choices are effective for the workloads tested.

Weaknesses

My evaluation focuses exclusively on the novelty of the contributions, and from this perspective, the paper's claims are significantly overstated. The constituent ideas presented as novel are, in fact, applications of well-established concepts from the domains of database systems, GPU computing, and fundamental data structures.

1. The HISA Data Structure is Not Fundamentally Novel: The central claimed contribution, HISA (Section 4, Page 4), is a composite structure. Its components are: a dense data array, a sorted index array over that data, and a hash table that maps join keys to starting offsets within the sorted index array. This is not a new data structure concept. The core mechanism—using a hash table to gain O(1) average-time access to the beginning of a sorted run of data—is a standard indexing pattern. More pointedly, the authors themselves state that HISA is "inspired by HashGraph [15]". HashGraph uses a hash table to map vertex IDs to their starting offset in a contiguous edge list array. HISA applies the exact same principle: mapping a join key (analogous to a vertex ID) to its starting offset in a contiguous, sorted tuple array (analogous to an edge list). The delta here is the application domain (relational joins vs. graph traversal), not the invention of a new data structuring principle. It is an effective adaptation, but not a novel structure.

2. "Novel Strategies" are Standard Practice: The two other strategies claimed as novel are also applications of existing principles.

   • Temporarily-Materialized n-way Joins (Section 5.2, Page 7): The strategy of decomposing a multi-way join into a sequence of binary joins with materialized intermediate results is a classic technique in query optimization known as pipeline breaking. In the specific context of GPUs, decomposing a complex, irregular kernel into a sequence of simpler, more regular kernels is a canonical pattern for managing thread divergence and improving warp occupancy. This is not a new idea, but rather a standard and necessary GPU programming practice applied to the Datalog domain.
   • Eager Buffer Management (EBM) (Section 5.3, Page 8): The described technique of allocating a buffer with a size of full + k * delta is functionally identical to the amortized reallocation strategy used by dynamic array implementations (e.g., C++'s std::vector) for decades. When capacity is exceeded, a new, larger block of memory is allocated (often 1.5x or 2x the old size) to reduce the frequency of future reallocations. Calling this a novel strategy in the context of Datalog buffer management overlooks its deep roots as a fundamental algorithm for dynamic memory management.

In summary, the contribution of this paper is one of excellent engineering, not of novel discovery. The authors have successfully built a fast system by correctly identifying and applying the right tools from the existing computer science toolbox. However, the work does not introduce new algorithms or data structures in a way that would expand that toolbox.

Questions to Address In Rebuttal

The authors should use the rebuttal to clarify and defend their claims of novelty with respect to prior art.

1. Please clarify the substantive conceptual delta between the HISA data structure's core mechanism (hashing to an index to begin a range scan) and the approach used in prior work such as HashGraph [15]. Is the claimed novelty primarily in the engineering and application to relational data, or is there a more fundamental algorithmic or structural difference that has been overlooked in this review?

2. The strategy of temporary materialization for n-way joins is presented as a novel optimization. Could the authors position this against the common practice in the GPU query processing literature of decomposing complex operations into sequences of regular kernels to manage SIMT divergence? Please explain what makes this application to Datalog distinct and novel from this established pattern.

3. The Eager Buffer Management (EBM) technique appears to be a direct application of the well-known amortized growth strategy for dynamic arrays. Could the authors please clarify what makes this a novel research contribution, rather than the application of a standard computer science principle to a new system?

Title: Optimizing Quantum Circuits, Fast and Slow Reviewer: The Guardian

---

Summary

The authors present GUOQ, a stochastic optimization algorithm for quantum circuits. The method is an application of simulated annealing that operates over a set of "circuit transformations" which unify two distinct optimization strategies: fast, local rewrite rules and slow, global unitary resynthesis. The latter is permitted to introduce a bounded degree of approximation error. The central claim is that this simple, unified approach significantly outperforms a wide range of state-of-the-art quantum circuit optimizers on benchmarks for both NISQ and FTQC architectures, primarily by achieving superior 2-qubit gate count reduction.

Strengths

1. Empirical Results: The numerical results presented, particularly for NISQ-era gate sets (Figure 8, page 8), are impressive on their face. The reported average 2-qubit gate reduction of 28% surpasses other superoptimizers and standard compilers by a significant margin under the specified experimental conditions.

2. Ablation Studies: The paper provides a clear justification for its hybrid approach through effective ablation studies. Figure 10 (page 9) convincingly demonstrates that the synergistic combination of rewrite rules and resynthesis is more powerful than either strategy in isolation. Furthermore, the analysis in Q3 (Figure 11, page 10) provides evidence that the authors' stochastic search is more effective than more structured approaches like sequential application or beam search for this problem.

3. Conceptual Framework: The abstraction of both rewriting and resynthesis into a common "circuit transformation" τ_ε (Section 4.1, page 5) is a clean and logical formalism. It provides a sound basis for composing approximate and exact operations, with the error bound proof (Theorem 4.2) providing a necessary, albeit straightforward, theoretical guarantee.

Weaknesses

My primary concerns with this work lie in the methodology's justification, the fairness of the experimental comparison, and the strength of the claims, particularly regarding the FTQC regime.

1. Methodological Novelty vs. Hyperparameter Tuning: The authors describe their algorithm as "radically simple." This is accurate; it is a direct application of simulated annealing, a well-established metaheuristic from the 1980s. The novelty therefore does not lie in the search algorithm itself but in its application. However, the performance of such an algorithm is critically dependent on its hyperparameters. The paper provides scant justification for its key choices:

   • The 1.5% probability of choosing the expensive resynthesis operation (Section 5.3, page 7) is presented as a fixed value with no justification.
   • The temperature parameter t is set to 10 based on an "empirical sweep" (Section Q1, page 8). These appear to be magic numbers. Without a thorough sensitivity analysis, it is impossible to know if the reported success is a general property of the method or the result of careful tuning to this specific benchmark suite. This raises serious questions about the method's robustness and generality.

2. Incongruous Experimental Comparison: The evaluation in Q1 compares GUOQ against a heterogeneous set of tools under a uniform one-hour time limit. This is a fundamentally flawed comparison. Tools like Qiskit and TKET are production compilers designed to produce reasonable results in seconds, not engage in hour-long superoptimization. Comparing them in this regime is an apples-to-oranges comparison that inflates GUOQ's apparent superiority. The only truly fair competitors in this setup are other superoptimizers like BQSKit, QUESO, and Quarl. While GUOQ performs well against them, the narrative of outperforming the entire field is based on this mismatched framing.

3. Overstated Claims in the FTQC Regime: The paper's claims of strong performance are significantly weakened in the FTQC setting. The primary goal for FTQC optimization is T-gate reduction, as T-gates are extremely expensive to implement fault-tolerantly. Figure 12 (page 11) clearly shows that the specialized optimizer PyZX substantially outperforms GUOQ on this critical metric. The authors concede this and pivot to showing that GUOQ can be used as a post-processing step to PyZX to reduce the secondary CX-gate metric (Figure 14, page 11). While this is a useful result, it directly contradicts the paper's central narrative of providing a single, unified framework that is broadly superior. The framework is demonstrably not the best for the most important FTQC optimization task.

4. Superficial Treatment of Approximation: The framework treats resynthesis as a black box that returns a circuit with some error ε. Theorem 4.2 provides a simple additive bound on this error. However, this treatment is superficial. It fails to discuss how accumulating approximation error might interact with the optimization process itself. For instance, an approximate subcircuit may no longer syntactically match the pattern of an exact rewrite rule, potentially closing off optimization pathways. The dynamics of searching through a space of approximate circuits are more complex than the paper acknowledges.

Questions to Address In Rebuttal

1. Please provide a sensitivity analysis for the key hyperparameters: the 1.5% resynthesis probability and the temperature t=10. How does the performance of GUOQ degrade as these values are perturbed? This is essential to establish the robustness of the method beyond the specific benchmarks used.

2. Can the authors justify the one-hour time limit comparison against fast production compilers? To make a more compelling case, please provide results comparing GUOQ to Qiskit and TKET at time scales relevant to their intended use (e.g., 10 seconds, 1 minute).

3. Please clarify the paper's primary claim in light of the FTQC results. Is GUOQ a comprehensive, unified optimizer that should be chosen over other tools, or is it a complementary technique that is best used in a pipeline with specialized optimizers like PyZX? The current framing is contradictory.

4. Could the authors elaborate on the interaction between approximate and exact transformations? Does the introduction of approximation error ε via resynthesis ever prevent the subsequent application of beneficial, exact rewrite rules that would have otherwise matched? Does your search algorithm account for this possibility?

Paper Title: Optimizing Quantum Circuits, Fast and Slow Reviewer Persona: The Synthesizer (Contextual Analyst)

---

Summary

This paper presents GUOQ, a novel approach to quantum circuit optimization that elegantly unifies two traditionally disparate techniques: fast, local rewrite rules and slow, globally-aware unitary resynthesis. The core contribution is twofold. First, the authors introduce a clean theoretical framework that abstracts both optimization methods into a single concept: an (ε)-approximate circuit transformation. This provides a common language and allows for a formal analysis of error accumulation. Second, they propose a "radically simple" search algorithm inspired by simulated annealing to apply these transformations. This stochastic, lightweight approach randomly alternates between fast and slow optimization techniques on random subcircuits.

The central thesis, supported by an extensive empirical evaluation, is that this simple, randomized search strategy significantly outperforms more complex and structured state-of-the-art optimizers, including industrial toolkits like Qiskit/TKET and specialized superoptimizers like Quarl. The work makes a compelling "bitter lesson" argument for the power of scalable search over intricate, hand-crafted heuristics in the complex landscape of quantum circuit optimization.

---

Strengths

1. Conceptual Elegance and Unification: The paper's primary strength is its beautiful and effective unification of rewriting and resynthesis. By abstracting both into the transformation concept (Section 4.1, page 5), the authors provide a powerful and flexible framework. This is a significant intellectual contribution that simplifies the conceptual model of circuit optimization and allows disparate methods to be combined in a principled way.

2. A Compelling "Bitter Lesson" for Quantum Compilation: The work positions itself as an example of Rich Sutton's "bitter lesson," where simple methods that leverage computation (in this case, rapid random search) ultimately outperform more complex approaches based on human-designed heuristics or learning policies. The stunningly strong results against Quarl—a sophisticated reinforcement learning-based optimizer that requires a GPU (Figure 1, page 1; Figure 8, page 8)—provide powerful evidence for this claim. This is an important and timely perspective for the quantum compilation community, which is at risk of developing increasingly complex and brittle optimization strategies.

3. Extensive and Convincing Empirical Evaluation: The evaluation is thorough and robust. The authors compare GUOQ against seven state-of-the-art tools on a diverse benchmark suite of 247 circuits. They consider multiple gate sets (IBMQ, IonQ, etc.) and crucially, address optimization objectives for both the NISQ and FTQC eras (Sections Q1-Q4). The results are not just incrementally better; they are substantially and consistently superior across most benchmarks and metrics. The ablation study in Q2 (Figure 10, page 9) effectively demonstrates the synergy between rewriting and resynthesis, showing that the whole is truly greater than the sum of its parts.

4. Flexibility and Synergy with Existing Tools: The framework is not presented as a monolithic replacement for all other tools, but as a flexible engine. The experiment in Q4 where GUOQ is used to post-process the output of PyZX is particularly insightful (Figure 14, page 11). It shows that GUOQ can drastically reduce the CX-count of circuits already optimized for T-count by a domain-specific tool. This demonstrates its potential as a powerful, general-purpose component in a larger, multi-stage compilation toolchain, capable of optimizing for multifaceted objectives.

---

Weaknesses

While the paper is excellent, a few points could be strengthened to further solidify its contribution:

1. Analysis of the Search Landscape: The paper convincingly shows that the randomized approach works well, but could benefit from a more explicit discussion of why. The optimization landscape for quantum circuits is likely vast, non-convex, and riddled with local minima. A structured search (like beam search in GUOQ-BEAM, Figure 11, page 10) or a fixed sequence of passes (like in Qiskit) can easily get trapped. The authors hint that resynthesis allows the search to "teleport" to new regions of the solution space (Section 3, page 5). A more direct and detailed discussion of this intuition would elevate the paper's theoretical contribution beyond just the empirical results.

2. Sensitivity to Hyperparameters: The algorithm's simplicity is a major strength, but it relies on a few key hyperparameters, namely the simulated annealing temperature t and the probability of choosing resynthesis (set to 1.5% in Section 5.3, page 7). The paper mentions that t=10 was chosen empirically, but a brief discussion on the sensitivity of the results to these choices would be valuable. Is the performance robust across a range of values, or is it highly tuned? This is important for understanding the algorithm's practicality.

3. Future Scalability of Random Subcircuit Selection: The current strategy of random subcircuit selection appears highly effective. However, as quantum circuits scale to millions of gates and thousands of qubits, one can imagine that purely random sampling might become inefficient, spending most of its time on non-optimizable regions. While beyond the scope of this paper's evaluation, a brief discussion on potential future scaling challenges and whether a hybrid approach (e.g., lightweight heuristics to bias the "random" selection) might eventually be necessary would show foresight.

---

Questions to Address In Rebuttal

1. The comparison against GUOQ-BEAM (Figure 11, page 10) is very interesting. It suggests that making many rapid, stochastic moves is more effective than carefully cultivating a smaller set of high-quality candidates. Could you elaborate on your intuition for why beam search, a powerful technique in many other search domains, falls short here? Does this reveal something fundamental about the structure of the quantum circuit optimization problem?

2. Could you comment on the sensitivity of GUOQ's performance to its key hyperparameters, specifically the simulated annealing temperature t and the 1.5% probability of invoking resynthesis? How does the balance between "fast" and "slow" moves affect convergence, and is there a risk of "over-exploring" with resynthesis if the probability is set too high?

3. The paper makes a compelling argument for the power of simple, randomized search today. Looking ahead to the fault-tolerant era with potentially millions of gates, do you foresee any fundamental scaling limitations to the random subcircuit selection strategy? Would the "bitter lesson" still hold, or might there be a place for more structured or guided search techniques to work in concert with the stochastic engine you have proposed?

Reviewer: The Innovator (Novelty Specialist)

Summary

This paper presents GUOQ, a quantum circuit optimizer that aims to unify two distinct optimization strategies: fast, local rewrite rules and slow, global unitary resynthesis. The authors propose a formal framework that treats both strategies as abstract "circuit transformations" with an associated error ε. The core of their contribution is a "radically simple" algorithm based on simulated annealing to stochastically apply these transformations to subcircuits, searching for a lower-cost circuit configuration under a global error budget. The empirical results are strong, showing significant improvements in gate count reduction over several state-of-the-art optimizers.

The central novel claim is not the invention of a new optimization algorithm, but rather the insight that a well-established, simple stochastic search heuristic, when applied to a unified set of both fast (rewrite) and slow (resynthesis) transformations, can dramatically outperform more complex, domain-specific search strategies in quantum circuit optimization.

Strengths

The primary strength of this work lies in its compelling empirical demonstration. The authors show that a conceptually simple approach can outperform more sophisticated and computationally expensive methods, such as the reinforcement learning-based Quarl [32]. This is a valuable "bitter lesson" [62] style contribution for the quantum compiler community. The proposed framework for unifying transformations under a single abstraction with a composable error bound (Section 4) is clean and provides a solid foundation for the algorithm.

Weaknesses

My evaluation focuses exclusively on the novelty of the core ideas presented. From this perspective, the paper's contributions are more evolutionary than revolutionary. The core algorithmic machinery is a direct application of pre-existing and well-understood concepts.

1. The Core Algorithm is Not Novel: The proposed search strategy is explicitly described as being "inspired by simulated annealing" [28], a 40-year-old optimization heuristic. The process of maintaining a candidate solution, proposing a random "move" (in this case, applying a transformation τ_ε), and probabilistically accepting it based on a cost function is textbook simulated annealing or, more broadly, a Metropolis-Hastings style MCMC search.

2. Strong Precedent in Classical Superoptimization: The high-level concept of using stochastic MCMC search to find optimal program representations is the cornerstone of classical superoptimization. Specifically, STOKE [55] (Schkufza et al., 2013) uses MCMC to search the space of x86 assembly programs. STOKE's methodology is conceptually identical to GUOQ's: it randomly mutates a program (the "move") and uses a cost function and correctness checks (the "acceptance probability") to guide the search toward an optimal equivalent program. GUOQ can be accurately characterized as an instantiation of the STOKE methodology in the quantum domain, where the "mutations" are applications of rewrite rules or resynthesis instead of changes to opcodes and operands. While the domain and specific transformations are new, the foundational search paradigm is not. The paper mentions STOKE in its related work (Section 7), but it does not sufficiently address this deep conceptual overlap.

3. The Unifying Framework Builds Directly on Prior Art: The framework for composing transformations with error bounds is presented as a key contribution. However, the theoretical heavy lifting for composing approximate operations appears to be borrowed from prior work. The proof of Theorem 4.2, which provides the additive upper bound on error, explicitly relies on the analysis from Patel et al. [45] (QUEST). The novelty here is the abstraction of packaging both exact rewrite rules (ε=0) and approximate resynthesis (ε>0) into this existing error-composition model, which is a useful engineering abstraction but not a fundamental theoretical advance.

The "delta" between this work and the prior art is therefore the application of a classical stochastic superoptimization strategy to a new set of operators (quantum rewrite rules and resynthesis) in the quantum domain. The key contribution is the insight that this simple, non-novel combination is highly effective, not the invention of the combination itself.

Questions to Address In Rebuttal

1. The authors frame GUOQ as a "radically simple algorithm." Given the direct mapping to simulated annealing and the strong conceptual precedent set by MCMC-based classical superoptimizers like STOKE [55], could the authors more precisely articulate what, if any, part of the core search algorithm is novel? Please position your work more directly against STOKE and clarify why it should be considered more than a straightforward application of that search paradigm to the quantum domain.

2. The paper claims to "unify two disparate ideas" (Abstract, page 1). Is the novelty in the idea of combining them, or in the specific method of randomized interleaving that the simulated annealing approach provides? Many compilers combine disparate optimization techniques using a fixed pass schedule. Is the primary insight here that for quantum circuits, a randomized schedule is superior to a deterministic one?

3. The ablation study in Q3 (Figure 11, page 10) compares GUOQ to fixed-sequence and beam-search alternatives. This convincingly shows GUOQ's search strategy is effective. Does this effectiveness stem primarily from the tight, random interleaving of fast and slow operations, or from the stochastic nature of the search which allows it to escape local minima that trap deterministic or more greedy approaches like beam search? A clearer diagnosis of why this well-known algorithm works so well here would strengthen the paper's contribution.

Paper Title: PartIR: Composing SPMD Partitioning Strategies for Machine Learning Reviewer: The Guardian

---

Summary

The paper presents PartIR, a compiler framework for partitioning large-scale neural network computations for SPMD execution. The central thesis is that partitioning strategies should be decoupled from the model's implementation. To this end, PartIR introduces a "schedule-like API" where users define a sequence of "tactics" (e.g., sharding a specific input tensor along a mesh axis). These tactics are incrementally applied as semantics-preserving rewrites on an MLIR-based Intermediate Representation (PartIR:Core), which utilizes functional loop and slice primitives. A core contribution is a propagation pass that extends initial sharding decisions throughout the computation graph, guided by a "Tile-Mapping Registry" (TMR). The system ultimately lowers these representations to device-local programs with explicit communication collectives. The authors claim PartIR is expressive, decoupled, and predictable, and they evaluate its performance against GSPMD on several models, showing comparable results.

Strengths

While maintaining a critical stance, I must concede a few well-argued points:

1. Well-Motivated Problem: The paper correctly identifies a significant pain point in large-scale ML: the entanglement of model logic with complex, hardware-specific parallelism annotations. The goal of decoupling these concerns is a valid and important research direction.

2. Incremental Rewriting for Conflict Resolution: The most compelling piece of evidence in the paper is presented in Section 7.4 and Figure 7. The comparison between PartIR and its single-tactic variant (PartIR-st) demonstrates that the incremental application and propagation of tactics is critical for resolving sharding conflicts that would otherwise lead to out-of-memory errors or suboptimal performance. This provides strong justification for the core architectural choice of the system.

3. Predictability of Collectives: The analysis in Table 3 (Section 7.3) successfully supports the claim of predictability. By showing a direct correspondence between the high-level strategies (e.g., BP, BP+MP+Z3) and the number of generated communication collectives, the authors demonstrate that their system behaves as a user would analytically expect, which is a notable improvement over opaque, heuristic-driven systems.

Weaknesses

My primary responsibility is to ensure the rigor of published work. The current manuscript contains several significant flaws and overstated claims that undermine its conclusions.

1. The "Decoupling" Claim is Contradicted by Escape Hatches: The central premise of the paper—a clean separation of the ML implementation from the partitioning strategy—is critically weakened by the admissions in Section 8. The introduction of atomic actions to prevent propagation and the tag primitive to name and force replication of intermediate tensors are, for all intents and purposes, model-internal annotations. The transpose example on page 806 is a clear case where the partitioning strategy fails and requires the user to modify the program's structure. The paper provides no data on how frequently these "escape hatches" are needed in real-world models. If they are common, the primary contribution of "decoupling" is more of an ideal than a reality.

2. Crucial Limitations Presented as Minor Issues: The discussion in Section 8 dismisses the lack of robust reshape support and the inability to handle uneven sharding (requiring padding) as simple limitations. This is a severe mischaracterization. Reshapes and complex tensor manipulations are ubiquitous in modern architectures, especially Transformers. A partitioning system whose propagation logic "gets blocked" by such a fundamental operation is not general-purpose. This limitation suggests the loop-based rewrite system is too rigid. The failure to address this suggests the system is only proven to work on a curated set of well-behaved models.

3. Performance Results Lack a Compelling Argument for Adoption: The evaluation in Section 7.2 (Table 2) concludes that PartIR achieves performance that is "on par with that of GSPMD, with negligible differences." While demonstrating parity with a strong baseline is necessary, it is not sufficient. The paper proposes a new, complex compiler stack. For the community to adopt it, there must be a clear advantage. Since the performance is merely equivalent, the argument must pivot to superior ergonomics and programmability. However, the paper presents no user studies, case studies of developer productivity, or other qualitative evidence to substantiate this implicit claim. Without a demonstrated advantage in either performance or usability, the rationale for PartIR's existence is weak.

4. The Tile-Mapping Registry (TMR) is Opaque and Unassessed: The entire propagation engine (Section 5.2.2) relies on the TMR. This registry is presented as a set of algebraic rules that define how sharding propagates through operations. However, the paper provides only trivial examples (matmul, add). The complexity, scalability, and maintainability of this TMR for the full set of StableHLO operations are never discussed. How are new or custom ops handled? Is this registry manually curated? An incomplete TMR would lead to propagation failure or, worse, silently suboptimal partitioning. The system's robustness is entirely dependent on this unexamined component.

5. Unsubstantiated Claim of Formal Verification: The paper states, "The critical transformation from PartIR:Core to PartIR:HLO is formally verified but omitted for brevity" (page 795). In a peer-reviewed academic publication, such a strong claim cannot be made without evidence. A formal proof is a significant contribution in itself. Without a proof sketch, a summary of the formal model, or at the very least a reference to an appendix or technical report, this claim is baseless and must be disregarded by the reader.

Questions to Address In Rebuttal

The authors must address the following points directly in their rebuttal:

1. On Decoupling: Please quantify the necessity of the tag and atomic primitives. For the models benchmarked in this paper (U-Net, T32, T48, GNS), how many instances of these model-internal modifications were required to achieve the reported results?

2. On Generality: Given the fundamental limitations regarding reshape operations, can you precisely define the class of models for which PartIR's propagation is guaranteed to succeed? How does your approach compare to GSPMD's ability to handle reshapes by manipulating logical device ID mappings, and why was a more limited approach chosen?

3. On TMR Scalability: What is the engineering effort required to define TMR rules for the entire StableHLO op-set? Provide an example of the TMR entry for a non-trivial operation, such as a convolution with complex padding attributes or a fused attention kernel, and discuss the challenges in defining its propagation rules.

4. On Justification for Adoption: If performance is on-par with GSPMD, what is the precise, evidence-backed argument for adopting PartIR? If the argument is superior programmability, why was a formal user study not conducted to validate this claim?

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper introduces PartIR, a system for partitioning large neural network models for distributed training. The core contribution is not merely another partitioning tool, but a fundamental reframing of the problem itself. The authors propose decoupling the parallelization strategy from the model's implementation, drawing inspiration from schedule-based compilers in high-performance computing and image processing (e.g., Halide).

This decoupling is achieved via a "schedule," a sequence of composable "tactics" that incrementally rewrite the program's Intermediate Representation (IR). The system is built on MLIR and uses a series of well-defined dialects (PartIR:Core, PartIR:HLO) to abstract parallelism first through functional loops and later through concrete SPMD collectives. This principled, rewriting-based approach aims to be more expressive, predictable, and maintainable than existing methods that rely on in-code annotations (like GSPMD) or opaque, monolithic automatic search. The paper provides a strong evaluation showing that this approach achieves performance comparable to the state-of-the-art while offering significant advantages in debuggability and modularity.

Strengths

The true strength of this paper lies in its elegant conceptual foundation and its connection to a rich history of compiler research.

1. The "Algorithm/Schedule" Dichotomy for ML Parallelism: The most significant contribution is the successful application of the algorithm/schedule separation, famously pioneered by Halide, to the domain of distributed ML training. By treating the partitioning strategy as a first-class, composable artifact (the schedule), the authors create a powerful separation of concerns. ML engineers can focus on model architecture, while systems performance experts can focus on crafting optimal distribution strategies for different hardware backends without modifying the model code. This is a profound and much-needed shift that addresses the growing problem of model portability and maintainability described in the introduction (Section 1, page 1).

2. Predictability through Incremental, Semantics-Preserving Rewrites: The system's design eschews complex, heuristic-based conflict resolution in favor of an ordered, incremental application of tactics. As shown in the discussion on conflicts (Section 5.2.3, page 8) and the evaluation in Section 7.4 (Figure 7, page 12), applying strategies sequentially allows for the natural prioritization of decisions (e.g., batch parallelism before parameter sharding), resolving potential conflicts in a predictable manner. This stands in stark contrast to annotation-based systems where conflicting annotations can lead to difficult-to-debug performance issues. The fact that users can inspect the IR after each tactic is a massive leap forward for the debuggability of complex parallel schemes. The results in Table 3 (page 11), which show the expected number of collectives for given strategies, provide strong evidence of this predictability.

3. A Principled, Multi-Level IR Abstraction: The compiler architecture (Figure 3, page 5) is well-conceived. The initial lowering to PartIR:Core, which represents parallelism as functional loop and slice operations, is particularly insightful. It allows the system to reason about tiling and data distribution algebraically via the Tile-Mapping Registry (TMR), independent of the final SPMD execution model. This formal approach is more robust and extensible than the pattern-matching on low-level collectives that other systems are forced to employ, as discussed in the critique of GSPMD in Section 8 (page 12).

Weaknesses

The paper is strong, and the weaknesses identified are more about the boundaries of the contribution and opportunities for deeper exploration than fundamental flaws.

1. Scope of Expressiveness and the "Reshape Problem": The paper rightly identifies that its rewriting system based on tiling and propagation faces challenges with complex data layout transformations, particularly reshapes (Section 8, page 12). This is a classic challenge for this style of compiler. While GSPMD's approach of manipulating logical device IDs is acknowledged as more flexible here, it comes at the cost of the brittleness the authors critique. The work would be stronger if it discussed potential paths forward. Could the schedule API be extended to include explicit "mesh reshaping" tactics? Or does this problem point to a fundamental limitation of the loop/slice abstraction for certain classes of computation?

2. The Complexity Shift: From Annotations to Schedules: While the paper successfully argues for decoupling, one could argue it shifts complexity from writing correct in-code annotations to writing correct schedule programs. The AutomaticPartition tactic is presented as a solution (Section 3, page 5), but its interplay with manual tactics is not fully explored. For a user, it is not immediately clear how one would debug a situation where a manual tactic and an automatic one lead to a suboptimal, combined strategy. The paper would benefit from a more detailed discussion of the "ergonomics" of composing manual and automatic search within the PartIR framework.

3. Positioning Relative to its Successor: The authors note that the learnings from PartIR have been incorporated into Shardy, a joint open-source project (Section 2.1, page 2). While this transparency is commendable, it leaves the reader wondering about the precise nature of this paper's contribution in the current landscape. A clearer delineation of which core PartIR concepts were "proven" and carried forward, and which were superseded by different ideas in Shardy, would help contextualize the lasting impact of this work.

Questions to Address In Rebuttal

1. Regarding the limitations with reshapes, could the authors elaborate on whether they see a path to supporting these transformations within the PartIR philosophy of semantics-preserving rewrites, or if this class of problem inherently requires a different abstraction?

2. Could you provide more insight into the composition of manual and automatic tactics? Specifically, how does the system handle propagation and potential conflicts when an automatic tactic is introduced into a schedule? For instance, what happens if AutomaticPartition on axis "M" proposes a sharding that conflicts with a user's prior manual sharding on axis "B"?

3. Could you please clarify the intellectual lineage from PartIR to Shardy? What are the one or two most critical design principles from PartIR that were validated and adopted by Shardy, and what was the primary limitation in PartIR that motivated a different approach in Shardy's design? This would greatly help the committee assess the impact of this specific paper.

Reviewer: The Innovator (Novelty Specialist)

Summary

The paper presents PartIR, an MLIR-based compiler framework for partitioning large machine learning models for Single-Program, Multiple-Data (SPMD) execution. The central claim is that by decoupling the partitioning strategy from the model's implementation, PartIR offers a more expressive, predictable, and debuggable system. The core mechanism involves expressing partitioning strategies as a "schedule" of "tactics." Each tactic in the schedule triggers an incremental rewrite of the program's intermediate representation (IR), followed by a rule-based propagation pass that distributes sharding decisions throughout the computation graph. This approach is positioned as an alternative to monolithic, annotation-based systems like GSPMD. The authors use a layered set of MLIR dialects (PartIR:Core, PartIR:HLO) to formalize this process, translating high-level tiling loops into low-level SPMD communication collectives.

Strengths

The primary novel contribution of this work lies in the specific application of a well-known paradigm—schedule-based compilation—to the problem of whole-program SPMD partitioning for ML models, and the use of incrementality as a conflict resolution mechanism.

1. A Novel Mechanism for Conflict Resolution: The most significant innovation is the use of a sequential, incremental schedule to predictably resolve sharding conflicts. In monolithic propagation systems, conflicting sharding decisions (e.g., sharding a tensor on the same dimension along multiple mesh axes) must be resolved with heuristics or user-provided annotations, which can be opaque and difficult to debug. By processing tactics sequentially and propagating their effects incrementally, PartIR provides an explicit ordering that resolves these conflicts by construction. The evaluation in Section 7.4 (page 11, Figure 7) provides compelling evidence that this incremental approach successfully partitions models where a monolithic version (PartIR-ST) fails due to memory exhaustion, demonstrating a clear benefit of this design.

2. Domain-Specific Adaptation of a Known Paradigm: While schedule-based compilation is not a new idea (see Weaknesses), its adaptation from single-device kernel generation (e.g., Halide) to multi-device, whole-program SPMD parallelism is non-trivial and represents a novel application. The introduction of PartIR:Core with its functional loop and slice operations provides a clean abstraction for representing parallel semantics over device meshes before committing to specific SPMD collectives.

Weaknesses

While the engineering is impressive, the work's core conceptual pillars are adaptations of long-established ideas from the programming languages and compilers literature. The novelty is more in the combination and application than in fundamental new principles.

1. The Core Abstraction is Not New: The central idea of separating an algorithm's definition from its optimization "schedule" is the foundational principle of systems like Halide [48], TVM [9], and others, as acknowledged in the related work section (Section 9, page 13). The paper's framing in the Abstract and Introduction could more clearly state that the novelty is not the paradigm itself, but its specific application and the benefits derived therefrom in the SPMD context. As it stands, the claims of "decoupling" might be misconstrued as a fundamental innovation of this work.

2. Rule-Based Propagation is Standard Compiler Practice: The propagation pass, based on the Tile-Mapping Registry (TMR) described in Section 5.2.1 (page 7), is a well-engineered implementation of semantics-preserving program rewriting. However, using algebraic properties of operations to propagate transformations is a cornerstone of compiler optimization. The TMR is essentially a manually curated database of rewrite rules for propagating tiling information. While effective, this is an evolutionary application of existing compiler technology rather than a revolutionary new technique.

3. Novelty at the Expense of Generality: The paper admits in Section 8 (pages 12-13) that the proposed abstraction has significant limitations, particularly with reshape operations. The loop-based tiling and propagation model struggles where the rank or layout of a tensor changes dramatically. In contrast, prior art like GSPMD [69] handles this by directly manipulating the mapping of logical device IDs to data shards, a more flexible if more complex approach. This suggests the novel abstraction in PartIR achieves predictability by sacrificing some of the generality found in existing systems.

Questions to Address In Rebuttal

1. The core concept of a schedule of tactics to guide compilation is central to systems like Halide [48] and TVM [9]. Can the authors more precisely articulate the novel scientific contribution beyond adapting this known paradigm from kernel generation to whole-program SPMD partitioning? The key seems to be incremental conflict resolution; I would encourage the authors to sharpen this as their primary contribution.

2. Section 8 notes that PartIR's propagation model struggles with reshape operations, a challenge that GSPMD [69] addresses. Does this limitation suggest that the core abstraction of program rewriting via loop tiling is fundamentally less powerful than manipulating data layouts directly, as in GSPMD? Please comment on this trade-off between the predictability of your system and the generality of prior art.

3. The Tile-Mapping Registry (TMR) in Section 5.2.1 appears to be a manually curated set of rewrite rules. How extensible is this registry to new or exotic operations not currently in your supported set? Is there a risk that the system's effectiveness is bottlenecked by the significant manual effort required to define these algebraic equivalences for the entire operator set of a modern ML framework?

Reviewer Persona: The Guardian (Adversarial Skeptic)

---

Summary

The paper presents PCcheck, a framework designed to mitigate the overhead of checkpointing in large-scale ML training. The central thesis is that existing mechanisms, such as CheckFreq and Gemini, are bottlenecked by their inability to handle more than one in-flight checkpoint at a time. To address this, PCcheck introduces support for multiple concurrent checkpoints, pipelining the process from GPU to CPU DRAM and then to persistent storage (SSD or PMEM) using multiple writer threads. The authors evaluate PCcheck across a range of models and claim it enables frequent checkpointing with minimal overhead (3%) and achieves significantly higher training goodput (up to 2.86x) in environments with frequent preemptions compared to state-of-the-art systems.

While the core idea is intuitive, the evaluation and analysis contain significant methodological weaknesses and unsubstantiated claims that call the paper's conclusions into question. The work rests on a narrow empirical foundation and fails to adequately address the systemic challenges introduced by its own design, particularly in a distributed context.

Strengths

1. The paper addresses a well-recognized and important problem in large-scale ML training: the tension between checkpoint frequency and performance overhead.
2. The core mechanism of decoupling checkpoint initiation from completion to allow for concurrent persistence is a logical next step in optimizing these pipelines.
3. The evaluation is broad in its choice of models, covering both vision and NLP tasks of varying sizes, and considers two different persistent storage media (SSD and PMEM).

Weaknesses

My primary concerns with this paper relate to the validity and generalizability of its core claims, which appear to be based on an oversimplified model of the problem and an evaluation that lacks sufficient rigor.

1. Over-generalization from a Single, Limited Failure Trace: The headline claim of achieving "up to 2.86× higher goodput" (Section 2, page 2 and Section 7, page 13) is derived from a simulation based on a single 16-hour preemption trace from a previous study [16]. This is a severe methodological flaw. Preemption patterns in cloud environments are notoriously variable and depend on time of day, data center load, and bidding strategies. A single trace cannot be considered representative of general spot instance behavior. The paper's strongest claim is therefore not generalizable and is only validated for one specific, arbitrary scenario. The authors fail to acknowledge this critical limitation.

2. Unsubstantiated Claims Regarding Distributed Coordination: The paper claims to support distributed training, but the mechanism and its overhead are dismissed with a hand-waving assertion. In Section 4.1 (page 8), the proposed coordination mechanism is described as a blocking operation where all peers report to rank 0 and wait for a notification to proceed. This is a synchronization barrier. The authors claim this "has negligible overhead compared to the actual training" (Section 3.2, page 5), but provide zero evidence to support this. In a large-scale system with frequent checkpointing (e.g., every 10 iterations), the latency of this network-bound barrier could easily become a dominant factor in the overall overhead, yet it is never measured or analyzed. This omission is a critical failure for a systems paper claiming applicability to distributed settings.

3. Inadequate Modeling of Resource Contention: The design of PCcheck inherently introduces contention for CPU memory bandwidth and, more importantly, storage I/O bandwidth. The analytical model presented in Section 3.4 (page 6) is an oversimplification that treats the time to write a checkpoint (Tw) as a constant. In reality, as the number of concurrent checkpoints (N) increases, Tw for any individual checkpoint will increase due to contention on the storage device. The model completely ignores this effect. The sensitivity study in Figure 12 (page 12) implicitly confirms this, showing diminishing or negative returns beyond 4 concurrent checkpoints. The paper fails to properly model or analyze the primary performance bottleneck that its own design creates.

4. Questionable Fidelity of Baseline Implementations: The comparison against Gemini [68] is based on the authors' own re-implementation, as the original is not open-source (Section 5.1, page 9). It is impossible for a reviewer to verify if this re-implementation is fair and optimized. Given that Gemini's performance is highly dependent on network conditions and its internal pipelining strategy, an unoptimized implementation could serve as a strawman. The poor performance of Gemini shown in Figure 8 could be an artifact of the implementation or the specific low-bandwidth network of the testbed, rather than a fundamental flaw in the Gemini design.

5. Inconsistent and Overstated Performance Claims: The abstract claims PCcheck maintains "minimal (3%) overhead," but the paper's own results do not consistently support this at high frequencies. For example, in Figure 8f (page 10), training BLOOM-7B with a checkpoint interval of 25 iterations shows throughput dropping from ~0.082 iters/sec to ~0.077 iters/sec, a slowdown of over 6%. While this is much better than the baselines, it is more than double the "3%" figure advertised in the abstract. Such claims should be stated with precise, qualified conditions, not as a blanket statement.

Questions to Address In Rebuttal

1. How can you justify the generalizability of the goodput results (Figure 9) when they are based on a single 16-hour preemption trace? Please provide evidence from other traces or a robust argument for why this specific trace is representative of the broader problem space.
2. Please provide empirical data measuring the overhead of your proposed distributed coordination mechanism (the blocking All-to-One and wait operation described in Section 4.1). How does this overhead scale with the number of nodes and the checkpointing frequency?
3. Your analytical model in Section 3.4 assumes Tw is independent of N. Your results in Section 5.4.1 suggest this is false. Please provide a more accurate model that accounts for storage bandwidth contention and validate it against your empirical results.
4. Please provide further details on your implementation of the Gemini baseline. Specifically, what steps were taken to ensure it was a faithful and optimized reproduction of the system described in the original paper? How does your network testbed compare to the one used in the Gemini paper?
5. Please clarify the exact conditions (model, checkpoint size, iteration time, checkpoint frequency) under which the claimed "minimal (3%) overhead" is achieved.

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper presents PCcheck, a system for persistent and concurrent checkpointing in large-scale machine learning (ML) training. The authors identify a critical bottleneck in existing fault-tolerance mechanisms: while frequent checkpointing is necessary to mitigate long recovery times (especially on unreliable resources like spot VMs), current systems introduce significant training overhead because they can only handle one checkpoint operation at a time. A new checkpoint request must wait for the previous one to be fully persisted to storage, causing the GPU to stall.

The core contribution of PCcheck is a well-engineered systems solution that decouples the training process from the persistence process by enabling multiple checkpoint operations to be in-flight simultaneously. By using a multi-buffered, pipelined architecture, PCcheck allows the GPU to continue training while previous checkpoints are being written to persistent storage (SSD or PMEM) in the background by multiple threads. The evaluation demonstrates that this approach dramatically reduces overhead, enabling checkpointing as frequently as every 10 iterations with only ~3% throughput degradation. More importantly, when simulated on a real-world preemption trace from a cloud provider, PCcheck achieves up to 2.86x higher "goodput" (useful training progress) compared to state-of-the-art baselines like CheckFreq and Gemini.

Strengths

1. Addresses a Highly Relevant and Economically Significant Problem: The paper is exceptionally well-motivated. The convergence of massive models, long training times, and the economic appeal of preemptible cloud instances has made efficient fault tolerance a first-order problem, not an afterthought. By framing the issue in terms of "goodput" on spot VMs (Figure 2, page 2), the authors connect their technical contribution directly to a tangible, real-world value proposition: reducing the cost and time of large-scale ML training.

2. Elegant Synthesis of Classic Systems Principles: The core idea behind PCcheck, while not fundamentally novel in the grand scheme of computer systems, is a fantastic example of applying established principles to a new and important domain. The use of concurrency, pipelining, and buffering to hide I/O latency is a cornerstone of high-performance systems, from database logging mechanisms (e.g., ARIES-style write-ahead logging) to decades of work in HPC checkpointing. The authors have successfully identified the specific data flow and bottlenecks of the ML training loop and tailored a solution that fits perfectly. It's a beautiful piece of systems engineering that bridges the gap between high-level ML frameworks and low-level hardware/OS primitives.

3. Strong Contextualization within the ML Systems Landscape: The paper does an excellent job of positioning PCcheck against its direct predecessors, CheckFreq and Gemini. The analysis that CheckFreq is bottlenecked by its single-checkpoint design and that Gemini is bottlenecked by network bandwidth in typical cloud environments (Section 5.2.1, page 10) is insightful. This demonstrates a clear understanding of the specific limitations of prior art and provides a compelling narrative for why a new approach—one that re-embraces and optimizes for local persistent storage—is necessary.

4. Comprehensive and Convincing Evaluation: The experimental methodology is a major strength. The use of a real-world preemption trace from André et al. [16] elevates the evaluation from a simple performance benchmark to a compelling simulation of real-world utility. Furthermore, the evaluation across multiple models (vision and NLP), scales (single-node to multi-node), and storage media (SSD and PMEM) demonstrates the robustness and general applicability of the proposed techniques.

Weaknesses

While the work is strong, its positioning could be broadened to acknowledge its deep roots in other fields, which would further strengthen its contribution.

1. Limited Connection to Broader Systems Literature: The paper primarily frames itself within the context of recent ML systems papers. However, the problem of concurrent, asynchronous checkpointing has been studied extensively in the HPC and Database communities. While the specific constraints of ML training (e.g., state is primarily model weights, GPU as the compute engine) are unique, explicitly connecting PCcheck's design to this broader history would help contextualize the work. For instance, the multi-buffered approach is reminiscent of classic double-buffering schemes used to overlap I/O and computation. Acknowledging these parallels would not diminish the novelty but rather show how enduring systems principles are being successfully adapted to the ML era.

2. Potential Underestimation of Resource Contention: The paper convincingly shows that PCcheck reduces GPU idle time. However, the cost of this is increased activity on other system resources: CPU cycles for the orchestrator and persistence threads, DRAM capacity for buffers, and contention on the PCIe bus and storage I/O channel. The authors briefly allude to this when discussing input-bound vision models (Section 4, page 7), but a more in-depth analysis of these trade-offs would be valuable. In a scenario where a training workload is already heavily utilizing the CPU for data preprocessing or the disk for dataset streaming, how does PCcheck's added load impact overall performance?

3. The Distributed Coordination Mechanism is Pragmatic but Simple: The proposed mechanism for distributed training relies on a rank 0 worker to coordinate a globally consistent checkpoint (Section 4.1, page 8). This is a practical and common solution, but it represents a potential single point of failure and a scalability bottleneck for training jobs running on thousands of nodes. The paper would benefit from a brief discussion of the limitations of this approach and potential avenues for more decentralized or robust coordination in future work.

Questions to Address In Rebuttal

1. Could the authors comment on the relationship between PCcheck's design and established techniques in HPC (e.g., multi-level checkpointing) or database recovery (e.g., asynchronous logging)? Acknowledging these connections could help place the work in a richer historical context.

2. The distributed coordination protocol relies on a rank 0 aggregator. Can the authors discuss the potential performance and fault-tolerance implications of this design choice as the number of nodes scales into the hundreds or thousands?

3. PCcheck offloads persistence work from the GPU to the CPU and storage subsystem. In workloads that are not purely GPU-bound (e.g., those with heavy data preprocessing on the CPU or streaming from disk), have you analyzed the potential for resource contention between the training pipeline and the PCcheck persistence threads? How does PCcheck manage or mitigate this?

Reviewer: The Innovator (Novelty Specialist)

---

Summary

This paper introduces PCcheck, a framework designed to enable frequent, low-overhead checkpointing for large-scale ML model training. The authors identify that existing state-of-the-art systems, such as CheckFreq and Gemini, become a bottleneck at high checkpointing frequencies because they can only handle one checkpoint operation at a time. The core claim of novelty in PCcheck is its ability to orchestrate multiple concurrent checkpoints in parallel. The system pipelines data from GPU to host DRAM and then uses multiple threads to persist the data to durable storage (SSD or PMEM), allowing training to continue with minimal stalling while several checkpoint operations are in flight. The authors demonstrate that this approach significantly improves training "goodput" in environments with frequent failures, such as those using preemptible cloud VMs.

Strengths

The primary strength of this work lies in its clear identification of a practical and increasingly important bottleneck in ML systems. The authors correctly diagnose the limitation of single in-flight checkpoint mechanisms (as depicted in Figure 4, page 3) and propose a direct solution. The experimental results, particularly the goodput evaluation using a real-world preemption trace (Figure 9, page 11), convincingly argue for the utility of the proposed mechanism, showing substantial gains over existing systems.

Weaknesses

My evaluation, focused exclusively on novelty, finds that the core conceptual contribution of this paper is limited. The central idea—using concurrent, asynchronous I/O to overlap computation with slow persistence operations—is a foundational pattern in high-performance computing and database systems, and is not a new invention.

1. Extensive Prior Art in Overlapping I/O and Computation: The principle of hiding I/O latency through concurrency is not novel. High-Performance Computing (HPC) has employed multi-level and asynchronous checkpointing for decades. Systems like SCR (Scalable Checkpoint/Restart) and VeloC explicitly use a multi-stage persistence pipeline (e.g., memory -> node-local SSD -> parallel file system) where data is migrated between stages asynchronously to minimize application stalls. While the authors' implementation is tailored to the GPU-DRAM-SSD pipeline of a modern ML server, the architectural pattern is functionally identical to what has long existed in the HPC domain. The novelty is therefore one of application and engineering, not a fundamental new concept.

2. Composition of Standard Techniques: The implementation of PCcheck, as described in Section 3 ("PCcheck Design"), appears to be a composition of well-known systems programming techniques. The use of lock-free queues to manage memory buffers for checkpoints and thread pools to perform parallel writes to storage are standard patterns for building high-performance I/O subsystems. While the orchestration of these components for the specific problem of ML checkpointing is the work of the authors, the underlying building blocks are not novel.

3. Delta Over Prior Art is Incremental: The key difference between PCcheck and its closest cited competitors (CheckFreq) is the move from N=1 concurrent checkpoint to N>1. While the performance benefits of this change are significant, the conceptual leap is small. It represents the next logical step in optimizing the pipeline rather than a paradigm shift in how checkpointing is performed. The paper essentially applies a more robust and scalable I/O pattern to a problem where previous solutions had used a simpler, more constrained version of that same pattern.

Questions to Address In Rebuttal

1. The core idea of managing multiple asynchronous I/O operations to hide latency is central to multi-level checkpointing in the HPC domain. Could the authors please articulate the fundamental conceptual difference between PCcheck's concurrent checkpointing and the asynchronous, multi-stage persistence pipelines used by established HPC fault-tolerance frameworks? Is the novelty purely in the application to the ML training loop, or is there a new algorithmic or architectural principle at play that I have missed?

2. The storage requirement for PCcheck is (N+1) * m (Table 1, page 5), which for a large m (e.g., BLOOM-7B's 108 GB checkpoint size) and N=4 concurrent checkpoints would require over 500 GB of fast persistent storage just for checkpointing. This seems to trade a performance problem for a potentially significant storage cost problem. Can the authors comment on whether this overhead makes the approach impractical for models that are orders of magnitude larger, and if the novelty of the approach is therefore limited to a specific scale of model?

3. The algorithm presented in Listing 1 (page 7) involves several synchronization primitives (atomic add, CAS, barriers) to manage the pointer to the latest consistent checkpoint. Given the complexity of concurrent programming, does this introduce new, subtle failure modes compared to the simpler single-checkpoint-at-a-time approach? Is the added complexity of correctness verification a significant trade-off for the performance benefit?

Paper Title: Performance Prediction of On-NIC Network Functions with Multi-Resource Contention and Traffic Awareness Reviewer Persona: The Guardian (Adversarial Skeptic)

---

Summary

This paper introduces Yala, a performance prediction framework for network functions (NFs) running on SmartNICs. The authors argue that existing frameworks, like SLOMO, are inadequate as they primarily focus on memory subsystem contention and have limited awareness of varying traffic profiles. Yala's proposed contributions are twofold: 1) a "divide-and-compose" model that separately models contention on hardware accelerators and the memory subsystem, and then composes these models based on an NF's execution pattern (pipeline or run-to-completion); and 2) a traffic-aware approach, including an adaptive profiling technique, to account for performance variations due to traffic attributes like flow count and packet contents. The evaluation, performed on BlueField-2 SmartNICs, claims significant improvements in prediction accuracy and reduction in SLA violations compared to the state-of-the-art.

While the paper presents a promising direction, I have significant concerns regarding the generalizability of its core modeling assumptions and the rigor of its comparative evaluation. The framework appears to rely on several simplifying assumptions that may not hold in the general case, and the impressive quantitative results may stem from a baseline comparison that is not entirely equitable.

Strengths

• Problem Significance: The paper correctly identifies a critical and timely problem. As SmartNICs become more powerful and are used to co-locate multiple NFs, understanding and predicting performance under multi-resource contention is paramount for efficient resource management and SLA adherence.
• Beyond Memory Contention: The attempt to model contention beyond just the memory subsystem, specifically including hardware accelerators, is a necessary step forward for the field. The paper rightly points out that this is a major blind spot in prior work.
• Pragmatic Profiling: The adaptive profiling technique described in Section 5.2 presents a pragmatic approach to mitigating the otherwise prohibitive cost of profiling across a high-dimensional space of traffic attributes.

Weaknesses

My primary concerns with this submission revolve around the robustness and generalizability of the core modeling choices and the fairness of the evaluation.

1. Over-simplification and Fragility of Accelerator Model: The entire accelerator contention model (Section 4.1.1, page 5) is predicated on the observation that the specific regex accelerator driver on their testbed platform uses a round-robin (RR) queuing discipline. This is a fragile assumption. What if other accelerators (e.g., compression, crypto) on the same NIC, or accelerators on different SmartNICs (e.g., from AMD Pensando, Intel), use different scheduling policies like weighted fair queuing or priority queues? The proposed queue-based model would be invalid. The paper presents this as a general approach, but provides evidence only for a single accelerator type on a single platform. This severely limits the claimed generalizability.

2. Post-Hoc Determination of Execution Pattern: The composition model (Section 4.2, page 6) is critically dependent on classifying an NF as either "pipeline" or "run-to-completion." The proposed method for this classification is alarming: "We resort to a simple testing procedure to detect an NF's execution pattern. We co-run the NF with our benchmark NFs, and see if Equation 2 or 3 fits its throughput drop better." This is not a predictive method; it is a post-hoc curve-fitting exercise. A robust model should be able to determine or infer this characteristic a priori. More importantly, real-world NFs are often complex hybrids of these two idealized patterns. The model provides no mechanism to handle such cases, which are likely the norm, not the exception. This fundamental weakness calls the entire "composition" approach into question.

3. Potentially Misleading Baseline Comparison: The paper claims a 78.8% improvement in accuracy over SLOMO [48]. However, SLOMO was designed primarily for memory contention for NFs running on host CPUs. The evaluation in this paper applies it to a problem space (multi-resource contention on an SoC with accelerators) for which it was not designed. Figure 7(a) (page 10) clearly shows SLOMO's error increasing with regex contention, which is entirely expected. While Yala is demonstrably better in this scenario, the magnitude of the improvement may be more of an indictment of applying a tool outside its domain than a testament to Yala's novel strengths. It is incumbent upon the authors to demonstrate that their adaptation of SLOMO represents the strongest possible state-of-the-art baseline, which is not evident.

4. Unspecified Hyperparameters and Sensitivity: The adaptive profiling method (Algorithm 1, page 8) relies on several hyperparameters (q, ε0, ε1, m). There is no discussion of how these were selected, nor any analysis of how sensitive the model's accuracy and the profiling cost are to their values. Without this analysis, the robustness of the profiling method is unknown. A different choice of hyperparameters could lead to significantly different results, potentially invalidating the conclusions drawn in Section 7.6.

Questions to Address In Rebuttal

1. Provide evidence or strong justification for why the round-robin queuing model for accelerators (Section 4.1.1) is applicable beyond the specific regex accelerator on the BlueField-2 platform. How would Yala's model adapt to accelerators with different, more complex scheduling policies (e.g., WFQ, strict priority)?
2. The method for determining an NF's execution pattern (Section 4.2) appears to be a post-hoc fitting exercise. How would Yala handle NFs that do not neatly fit the pure pipeline or run-to-completion models, or whose execution patterns might change dynamically with traffic?
3. Please justify why the version of SLOMO used for comparison, particularly its extension to traffic awareness via "sensitivity extrapolation," represents a fair and robust state-of-the-art baseline for the multi-resource and highly dynamic traffic scenarios evaluated. Did you consider alternative ways to adapt SLOMO that might have yielded a stronger baseline?
4. What was the methodology for selecting the hyperparameters (q, ε0, ε1, m) for the adaptive profiling algorithm (Algorithm 1), and how sensitive is the trade-off between profiling cost and model accuracy to these choices?
5. The model focuses on memory and specific accelerators. What about other potential sources of contention on a SmartNIC SoC, such as the on-chip interconnect/network-on-chip (NoC) or the PCIe bus bandwidth to the host? Have you quantified their impact, and can the framework be extended to incorporate them?

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper presents Yala, a performance prediction framework for network functions (NFs) co-located on modern SmartNICs. The authors identify a critical gap in existing work: prior models, like SLOMO, primarily focus on memory subsystem contention and lack sufficient awareness of dynamic traffic patterns, leading to poor accuracy in the complex SmartNIC environment. Yala's core contribution is a "divide-and-compose" methodology that addresses this gap. It divides the problem by creating separate, tailored models for each class of contended resource—a black-box, machine learning model for the complex memory subsystem and a white-box, queueing-based model for hardware accelerators. It then composes the outputs of these per-resource models based on the NF's execution pattern (pipeline vs. run-to-completion) to predict end-to-end throughput. The entire framework is augmented with traffic-awareness and an adaptive profiling strategy to manage data collection costs. The evaluation, conducted on BlueField-2 SmartNICs, demonstrates that Yala significantly outperforms the state-of-the-art, reducing prediction error by 78.8% and enabling use cases like scheduling and performance diagnosis with dramatically fewer SLA violations.

Strengths

This is an excellent systems paper that addresses a timely and increasingly important problem. Its primary strengths are:

1. Clear Motivation and Problem Framing: The paper does a superb job in Section 2 (pages 2-4) of demonstrating why existing solutions are insufficient. The empirical evidence showing the failure of a memory-only model (SLOMO) in the presence of accelerator contention (Figure 2a) and the model's brittleness to traffic variations (Figure 3b) provides a compelling and undeniable motivation for this work.

2. A Pragmatic and Insightful Hybrid Modeling Approach: The central idea of Yala is its "divide-and-compose" strategy, which is both elegant and effective. The decision to not force a single modeling technique onto the entire system is a key insight. Using a white-box, queueing-based model for the hardware accelerators (Section 4.1.1, page 5), based on the observation of their round-robin scheduling behavior, is a clever use of domain-specific knowledge where available. Conversely, pragmatically reusing a state-of-the-art black-box approach for the well-instrumented but complex memory subsystem (Section 4.1.2, page 6) is a sensible choice. This hybrid methodology represents a significant step forward from monolithic modeling approaches.

3. Contextualizing Performance within Application Structure: A standout contribution is the execution-pattern-based composition (Section 4.2, page 6). Recognizing that the impact of contention on one resource depends on whether the NF is structured as a pipeline or as a run-to-completion task is a crucial piece of the puzzle. This moves beyond simply modeling resource contention in isolation and begins to model how the system of the application and hardware interact, which is a more sophisticated and accurate view.

4. Strong and Convincing Evaluation: The empirical results are thorough and demonstrate substantial improvements over a strong baseline. The two use cases presented in Section 7.5 (page 10) are particularly compelling. The contention-aware scheduling scenario shows a tangible benefit, reducing SLA violations by over 90% compared to SLOMO. The performance diagnosis use case highlights Yala's ability to provide deeper insights, correctly identifying shifting bottlenecks where a simpler model would fail. This effectively elevates the work from a mere prediction tool to an enabler of smarter datacenter management.

Weaknesses

The weaknesses of the paper are more related to the boundaries of its exploration rather than fundamental flaws in the approach.

1. Simplified Abstraction of Execution Patterns: The binary classification of NFs into "pipeline" and "run-to-completion" is a powerful and effective simplification. However, real-world NFs and service chains can exhibit more complex, hybrid dataflow patterns. The paper would be strengthened by a discussion of the model's limitations in the face of such complexity and potential paths to extending the composition logic.

2. Uncertain Generalizability of the Accelerator Model: The white-box model for the regex accelerator is based on a specific, observed round-robin queueing discipline. While the authors demonstrate generalizability to another SoC-style SmartNIC (AMD Pensando, Section 8, page 12), the broader landscape of DPUs and IPUs may feature different accelerator architectures with more complex scheduling (e.g., priority queues, weighted fairness). The paper could better position its contribution by emphasizing the methodology of identifying and modeling the scheduling discipline, rather than the specific round-robin model itself.

3. Focus on Throughput Over Latency: The work is entirely focused on predicting maximum throughput. For many interactive or latency-sensitive NFs, tail latency is an equally, if not more, important SLA metric. While this is outside the paper's stated scope, a brief discussion on the challenges and possibilities of extending the Yala framework to predict latency would help contextualize its place in the broader performance landscape.

Questions to Address In Rebuttal

1. The composition model relies on a binary classification of execution patterns. How prevalent are more complex, hybrid patterns in real-world NFs? Could the authors comment on how Yala's framework might be extended to accommodate NFs that do not fit cleanly into the "pipeline" or "run-to-completion" molds?

2. The white-box accelerator model is a key strength. How critical is the round-robin scheduling assumption to this model's success? If faced with a SmartNIC employing a different policy (e.g., priority-based scheduling), would the general methodology of creating a white-box model still hold, simply requiring a different analytical formulation, or would a fundamentally new approach be needed?

3. Given that many NF SLAs are defined by latency targets, could the authors speculate on how their "divide-and-compose" framework could be adapted to predict latency metrics? What new per-resource data would be needed (e.g., modeling queueing delays instead of just service rates), and what would be the primary challenges in composing these per-resource latency predictions?

Reviewer: The Synthesizer (Contextual Analyst)

---

Summary

This paper presents Yala, a performance prediction framework for network functions (NFs) co-located on SmartNICs. The authors argue that existing frameworks, such as SLOMO, are inadequate for this environment because they primarily model memory subsystem contention and lack robust awareness of varying traffic patterns. Yala’s core proposed novelty is a "divide-and-compose" methodology. This involves creating separate performance models for different shared resources—specifically, a white-box queueing model for hardware accelerators and a black-box machine learning model for the memory subsystem. These individual models are then composed based on the NF's execution pattern (pipeline vs. run-to-completion) to predict the final end-to-end throughput. The framework also incorporates traffic attributes into the models and uses an adaptive profiling technique to manage the cost of data collection.

Strengths

The primary strength of this work lies in its novel synthesis of modeling techniques to address the specific and increasingly important problem of performance prediction on modern SmartNICs.

1. Identifies a Critical Gap in Prior Art: The paper correctly identifies that prior work on NF performance prediction (e.g., SLOMO [48], Bubble-Up [50]) is critically limited by its focus on memory contention on general-purpose CPUs. The extension to model contention on heterogeneous resources, particularly hardware accelerators, is a necessary and novel step for the SmartNIC domain. The authors effectively demonstrate this gap in Figure 2(a) (page 4).

2. Novel Hybrid Modeling Framework: The key novelty is the proposed hybrid framework itself. While the individual components are not new, their combination is. The choice to use a white-box, analytical queueing model for accelerators (where behavior is somewhat regular and observable via queues) and a black-box, ML-based model for the complex memory subsystem (where fine-grained performance counters are available) is a pragmatic and insightful design. This integration of disparate modeling paradigms into a single predictive system is the paper's main conceptual contribution.

3. Composition based on Execution Patterns: The explicit step of composing the per-resource models based on an NF's execution pattern (Section 4.2, page 6) is a significant advancement over simply summing or averaging the impacts of contention. While the underlying principles of pipeline vs. serial execution are fundamental, applying them to compose a hybrid set of contention models in this context is a novel and crucial part of the framework.

Weaknesses

From a novelty perspective, the work's primary weakness is that it is a clever synthesis of existing ideas rather than the invention of fundamentally new modeling techniques.

1. Constituent Components are Not Novel: The individual building blocks of Yala are well-established.

   • The use of gradient boosting regression with performance counters to model memory contention is directly inherited from SLOMO [48].
   • The use of queueing theory to model a resource with a round-robin scheduler (Section 4.1.1, page 5) is a classic and standard performance modeling technique.
   • The adaptive profiling algorithm (Section 5.2, page 8), which prunes insensitive dimensions and uses binary-search-style sampling, is a well-known heuristic in active learning and experimental design for reducing parameter space exploration.

2. The "Delta" is in the Combination, Not the Pieces: The paper could be clearer in positioning its novelty. The contribution is not a new ML algorithm or a new queueing theory result, but rather the architectural insight of how to combine them effectively for this specific problem domain. The current presentation sometimes blurs this line. The novelty is in the system design, which is valid, but it is not a fundamental algorithmic advance.

Questions to Address In Rebuttal

1. Generalizability of the Accelerator Model: The white-box model for the regex accelerator relies on the observation that it uses a round-robin (RR) queuing discipline (Section 4.1.1, page 5). How central is this specific scheduling policy to Yala's novelty? If a future SmartNIC employs a more complex or proprietary scheduling policy (e.g., weighted fair queuing, priority queues) for its accelerators, would the entire white-box modeling approach break down, or is the framework adaptable? Please clarify whether the contribution is the specific RR model or a more general methodology for modeling accelerators.

2. Prior Art on Hybrid Models: The core innovation appears to be the hybrid combination of an analytical model (for accelerators) and an empirical ML model (for memory). Can the authors cite the closest prior work in any systems domain (not limited to NFV) that has proposed such a hybrid, multi-resource performance model? A clear articulation of the delta between Yala and the closest related work in the broader performance modeling literature would strengthen the novelty claim.

3. Robustness of Execution Pattern Detection: The composition stage is critically dependent on correctly classifying an NF as either "pipeline" or "run-to-completion". The paper describes this as a "simple testing procedure" (Section 4.2, page 6). Complex NFs may exhibit hybrid patterns or behaviors that change with traffic load. How sensitive is the model's accuracy to misclassification of this pattern? The novelty of the composition approach is diminished if this classification step is fragile or ambiguous in practice.

Paper Title: PULSE: Accelerating Distributed Pointer-Traversals on Disaggregated Memory Reviewer: The Guardian

---

Summary

The authors present PULSE, a framework designed to accelerate pointer traversals in a rack-scale disaggregated memory environment. The system is composed of an iterator-based programming model, a novel accelerator architecture with disaggregated logic and memory pipelines, and an in-network distributed traversal mechanism leveraging a programmable switch. The authors claim that this design provides expressiveness, performance, energy efficiency, and scalable distributed execution, a combination they argue is missing from prior work. The paper includes an evaluation of an FPGA-based prototype against several baselines.

However, my review finds that several key claims regarding expressiveness, energy efficiency, and the practical benefits of its distributed traversal mechanism are not sufficiently substantiated by the provided evidence. The work rests on a foundation of speculative estimations and an evaluation whose scope does not fully support its ambitious "rack-scale" claims.

Strengths

1. Problem Motivation: The paper correctly identifies a critical and well-known performance bottleneck for disaggregated memory architectures: pointer-chasing workloads that exhibit poor locality and defeat traditional caching mechanisms.
2. Core Accelerator Insight: The architectural principle of disaggregating logic and memory pipelines (Section 4.2, Page 7) is sound. The justification, based on the observation that logic time (tc) is typically much shorter than memory fetch time (td) for these workloads, is logical and provides a solid basis for the hardware design.
3. Prototype Implementation: The development of a real-system FPGA prototype (Section 4.2, Page 9) is a commendable effort. It provides a degree of validation that is superior to simulation-only studies and grounds the performance results in reality, albeit with the caveats of a prototype.
4. Evaluation Breadth: The authors have compared PULSE against a reasonable set of baselines, including cache-only, CPU-based RPC, and ARM-based RPC systems, across three distinct and relevant real-world applications (Section 6, Page 10).

Weaknesses

1. Overstated "Expressiveness" of the Programming Model: The authors claim their iterator abstraction preserves expressiveness (Abstract, Section 1). However, the model imposes a severe "bounded computation" constraint, explicitly disallowing unbounded or data-dependent loops within an iteration (Section 3, Page 5). This fundamentally limits the complexity of logic that can be offloaded. While the provided examples (hash table, B+-tree) fit this model, it is highly questionable whether more complex graph traversal algorithms (e.g., those with unpredictable branching or nested loops) could be implemented without significant refactoring or falling back to the CPU. The claim of general expressiveness is therefore not supported.
2. Energy Claims are Fundamentally Unsubstantiated: The energy consumption analysis (Section 6.1, Figure 8, Page 12) is the most significant flaw in this paper. The PULSE-ASIC results are not measurements but rather estimations derived by scaling FPGA power numbers using a methodology from a nearly two-decade-old paper [95]. The validity of applying this 2006 methodology to modern process nodes and architectures is highly suspect. Furthermore, the energy figures for the RPC-ARM baseline are also an estimation, not a direct measurement. Relying on speculative estimations for two of the key comparison points undermines the entire energy efficiency claim and does not meet the standards of rigor for this conference.
3. Marginal End-to-End Benefit of the In-Network Traversal: The primary novelty of PULSE is its support for distributed traversals via a programmable switch. The authors claim this "cuts the network latency by half a round trip time" (Section 5, Page 9). However, the empirical evidence presented does not show a dramatic benefit. In the head-to-head comparison between PULSE and a variant that returns to the CPU (PULSE-ACC in Figure 9, Page 12), the end-to-end latency improvement is modest, appearing to be in the 15-20% range for the two-node case. While an improvement, it falls far short of the theoretical "halving" of network latency, suggesting that other overheads dominate or the benefit is less significant in practice than claimed. This discrepancy between the strong claim and the measured result is a critical issue.
4. Scalability is Assumed, Not Proven: The paper repeatedly refers to "rack-scale" deployments. Yet, the experimental evaluation is limited to a maximum of four memory nodes (Figure 7, Page 11). This is hardly rack-scale. The proposed hierarchical translation places a global address translation table on the switch (Figure 6, Page 9). The paper provides no analysis of the scalability of this centralized component. At a true rack scale with potentially hundreds of nodes and thousands of fine-grained memory allocations, this table's size, update overhead, and lookup latency could easily become a significant bottleneck. The authors have not provided any evidence that their design can scale beyond their small testbed.
5. Inconsistent Application of Baselines: In the evaluation, the Cache+RPC (AIFM) baseline is restricted to a single node for the B+-Tree workloads because, as the authors state, it "does not natively support... distributed execution" (Section 6.1, Page 10). While technically correct, this sidesteps a rigorous comparison. The purpose of a baseline is to establish a state-of-the-art comparison. By simply omitting the data points, the authors avoid demonstrating how much better PULSE is than a distributed version of AIFM, or discussing the complexity required to build one. This weakens the comparative power of the evaluation.

Questions to Address In Rebuttal

1. Regarding Expressiveness: Please provide a concrete example of a common pointer-traversal algorithm from a real-world application (e.g., in graph analytics or complex database indexes) that cannot be implemented within PULSE's bounded computation model. How do you justify the "expressive" label given this significant constraint?
2. Regarding Energy Claims: Please provide a robust justification for using a power scaling methodology from 2006 [95] to estimate ASIC performance for a modern system. Given the speculative nature of both the PULSE-ASIC and RPC-ARM energy figures, how can the paper's core claim of superior energy efficiency be considered valid?
3. Regarding Distributed Traversal Benefit: The measured end-to-end latency improvement from the in-network switch appears to be around 15-20% in Figure 9, not the 50% reduction in network time claimed in Section 5. Please provide a latency breakdown that reconciles the theoretical benefit with the measured, and much smaller, end-to-end improvement. What are the dominant overheads that are not accounted for in your high-level claim?
4. Regarding Scalability: The switch's global address translation table is a centralized resource. What is the upper bound on the number of distinct, fine-grained memory allocations this table can hold on current programmable switch hardware? At what scale (in terms of nodes or allocation frequency) would this table's capacity or update contention become the system's primary performance bottleneck? Your paper makes "rack-scale" claims that are not supported by the 4-node experiment; please provide evidence for this scalability.

Paper: PULSE: Accelerating Distributed Pointer-Traversals on Disaggregated Memory Reviewer Persona: The Synthesizer (Contextual Analyst)

---

Summary

This paper identifies and addresses a fundamental performance bottleneck in disaggregated memory architectures: the high latency of pointer-chasing traversals across the network. The authors correctly argue that traditional CPU-side caching is ineffective for these workloads due to poor data locality.

The core contribution is PULSE, a holistic, co-designed framework that offloads pointer traversals to lightweight accelerators at the memory nodes. The novelty of PULSE lies in its three synergistic components: 1. An expressive iterator-based programming model that provides a general abstraction for various linked data structures while constraining computation to make hardware acceleration tractable. 2. A novel disaggregated accelerator architecture where the logic and memory pipelines are decoupled and asymmetrically provisioned, efficiently matching the memory-bound nature of pointer chasing. 3. An in-network continuation mechanism that leverages a programmable switch to seamlessly route traversal requests between memory nodes, handling distributed traversals without costly round-trips to the initiating CPU.

The authors implement a prototype on FPGAs and a programmable switch, demonstrating significant end-to-end latency, throughput, and energy-efficiency gains (e.g., 9-34× lower latency than caching) for representative database, key-value store, and time-series workloads.

Strengths

This is an excellent systems paper that connects several important research threads into a cohesive and compelling solution.

1. Addresses a Critical and Well-Understood Problem: The "pointer-chasing problem" is a well-known Achilles' heel for any system that separates compute from memory, whether it's traditional NUMA or modern memory disaggregation. By focusing on this specific, high-impact problem, the work has immediate relevance and positions itself as a key enabler for the widespread adoption of disaggregated memory. The empirical motivation in Section 2 (Page 3, Fig 2) effectively illustrates the severity and prevalence of the problem.

2. Elegant, Principled Co-Design: The strength of PULSE is not in any single component, but in their synthesis.

   • The iterator abstraction (§3) is the right level of software interface. It maps to a familiar programming pattern, making it broadly applicable, while its "bounded computation" constraint is a pragmatic tradeoff that makes specialized hardware feasible.
   • The idea of a disaggregated accelerator (§4.2, Fig 4) is the paper's most insightful architectural contribution. Recognizing that the workload is asymmetric (brief computation tc, long memory wait td) and designing an accelerator with asymmetric resources (fewer logic pipelines, more memory pipelines) is a clever insight that directly attacks the core inefficiency of using general-purpose cores for this task.
   • Using the network switch for distributed continuations (§5) is an elegant solution to the distributed traversal problem. It reframes a distributed computation problem as a packet routing problem, leveraging the strengths of existing programmable network hardware and avoiding expensive CPU involvement.

3. Connects Multiple Research Domains: This work sits at a beautiful intersection of memory disaggregation, near-memory processing (NMP), and programmable networking. It takes the "offload computation" philosophy from NMP but proposes a far more resource-efficient and generalizable architecture than prior work. It uses the programmable network not just as a transport, but as an active component of the distributed execution model. This synthesis provides a valuable new perspective on how to build efficient, rack-scale computer systems.

4. Strong, End-to-End Evaluation: The evaluation is thorough and convincing. The authors build a real hardware prototype and compare it against a strong set of baselines, including caching, CPU-based RPC, and SmartNIC-based offloads. The results clearly demonstrate that their specialized, co-designed approach provides substantial benefits in performance and energy efficiency that cannot be achieved by any of the baseline approaches alone.

Weaknesses

My critiques are not focused on fundamental flaws but on exploring the boundaries of the proposed solution and its practical deployment.

1. Generality and Limits of the Programming Model: The iterator model is very powerful, but the paper primarily uses simple examples like key lookups. It would be beneficial to discuss the model's limitations more explicitly. How does PULSE handle more complex traversals, such as those that might need to dynamically modify the structure they are traversing (e.g., rebalancing a tree, splicing a list)? Are transactional semantics or atomic updates beyond the scope of this model? A deeper exploration of these edge cases would help define the practical application space.

2. System Complexity and Path to Adoption: PULSE is a holistic solution, which is a strength, but it also implies significant complexity. It requires a custom software toolchain (compiler from iterator to PULSE ISA), custom accelerators on memory nodes (FPGAs or ASICs), and a programmable network switch. This is a high barrier to entry. While the components are individually plausible, the paper could benefit from a brief discussion on the path to deployment. Could a subset of PULSE's benefits be achieved with, for example, just a SmartNIC-based accelerator without the programmable switch?

3. Assumptions about the Network Fabric: The in-network continuation model relies on a programmable switch with a global view of memory allocation (albeit at a coarse granularity). The paper could clarify the scalability of this aspect. How does the system handle dynamic re-partitioning or re-allocation of memory ranges across nodes? Is there a risk that the translation table or forwarding logic in the switch could become a bottleneck or a point of complexity in a large-scale, dynamic environment?

Questions to Address In Rebuttal

1. Could you elaborate on the expressiveness of the iterator model for more complex, state-modifying traversals? For example, could PULSE be used to implement an operation like find_and_move_to_front in a linked list, which requires modifying next pointers during the traversal? If so, how would state consistency be managed?

2. The distributed traversal mechanism is very elegant for reads. How does the model extend to handle distributed writes or atomic operations that might need to span multiple memory nodes? Does this necessarily require returning to the CPU for coordination?

3. Regarding the system's robustness, the paper mentions retransmission on timeout for requests from the CPU. What is the failure model for a distributed traversal that is already in progress? For instance, if a request is forwarded from Memory Node A to B, and Node B fails or drops the packet, how does the original CPU learn of this failure and what is the recovery process?

Reviewer: The Innovator (Novelty Specialist)

---

Summary

This paper presents PULSE, a framework designed to accelerate pointer-traversal workloads in a disaggregated memory environment. The authors identify that existing solutions, such as CPU-side caching or single-node near-memory processing (NMP), are insufficient for distributed linked data structures. The central claim of novelty rests on a three-part system design: (1) an iterator-based programming model to provide an expressive software interface, (2) a novel "disaggregated" accelerator architecture at each memory node that decouples logic and memory pipelines for efficiency, and (3) a mechanism using a programmable network switch to enable stateful traversals to continue seamlessly across multiple memory nodes. The authors implement and evaluate a prototype, demonstrating significant performance and energy efficiency gains over baseline and RPC-based approaches.

Strengths

From a novelty perspective, the paper's primary strength is the coherent system design for distributed stateful traversals. The core innovative concept is the use of a programmable network switch to act as a router for in-flight pointer-chasing operations (Section 5, page 9). While NMP for pointer traversals and the use of programmable switches for network offloads are individually established concepts, their synthesis here to solve the multi-node traversal problem is genuinely novel. The mechanism of packaging the iterator state (cur_ptr, scratch_pad) into a request that can be forwarded by the switch to the next memory node, without returning to the host CPU, is an elegant and previously unexplored solution to the single-node limitation of prior NMP accelerators.

A secondary, but still significant, novel contribution is the design of the PULSE accelerator itself (Section 4.2, page 7). The explicit decision to "disaggregate" the logic and memory pipelines within the accelerator (Figure 4, page 8) is a clever architectural insight. It directly addresses the memory-bound nature of these workloads, where tightly-coupled compute/memory resources in a traditional core design would lead to underutilization of the logic units. This design is a distinct and well-justified departure from both general-purpose cores (used in RPC schemes) and prior tightly-coupled specialized accelerators.

Weaknesses

The main weakness, in terms of novelty, is the framing of some well-established concepts as foundational contributions of this work.

1. The Iterator Programming Model: The paper presents the iterator-based interface (Section 3, page 5) as a key design element. While it is a good engineering choice for creating a flexible hardware-software contract, the iterator pattern itself is a cornerstone of software engineering and is by no means novel. The contribution here is its application as an interface for an NMP accelerator, which is an incremental step rather than a fundamental innovation.

2. The General Concept of NMP for Pointer Traversals: The paper correctly critiques prior work, but the idea of building specialized hardware close to memory to accelerate pointer chasing is not new. Seminal works like "Meet the Walkers" [90] and Hsieh et al. [76] explored this problem space in depth for in-memory databases and 3D-stacked memory, respectively. The authors' novelty is not in that they are building a pointer-chasing accelerator, but in the specific architecture of that accelerator (the disaggregated design) and, more importantly, its integration into a distributed system. The paper could be strengthened by more clearly positioning its work against these specific prior accelerator designs, rather than just against more generic RPC or caching systems. The current framing risks obscuring the true architectural novelty by re-litigating settled questions.

In essence, the novelty of PULSE is not in its individual conceptual building blocks (iterators, NMP, programmable switches) but in their specific and sophisticated synthesis to create a new system capability: efficient, rack-scale distributed pointer traversal. The paper should be more precise in claiming this system-level synthesis as its core contribution.

Questions to Address In Rebuttal

1. Comparison to Prior Specialized Accelerators: The evaluation primarily compares PULSE to systems using general-purpose cores (RPC/RPC-ARM). A more rigorous assessment of novelty would compare the PULSE accelerator's disaggregated design to a specialized but coupled design, such as the one proposed in "Meet the Walkers" [90]. Could the authors provide a conceptual analysis (or even a model-based estimation) of how their disaggregated architecture compares in terms of area, power, and performance for a single-node traversal against such a design? This would better isolate and justify the claimed benefits of the novel disaggregated pipeline.

2. Scope of the Iterator Abstraction: The scratch_pad provides a mechanism for stateful traversals. However, its fixed size appears to be a limitation. How does the PULSE model handle traversals where the intermediate state is unbounded or grows unpredictably, such as a Breadth-First Search (BFS) on a graph where the queue of nodes to visit can become very large? Does this represent a fundamental limit to the novelty of the approach, confining it to traversals with small, constant-sized state?

3. Scalability of the Switch-Based Routing: The novel distributed traversal mechanism relies on a translation table in the programmable switch (Figure 6, page 9). This table maps virtual address ranges to physical memory nodes. What are the scalability limits of this approach? As the number of memory nodes and the granularity of allocations increase, this table could grow beyond the capacity of on-switch memory. Is the novelty of this mechanism constrained to a rack-scale system, or do the authors envision a path for scaling it further?

Paper Title: QECC-Synth: A Layout Synthesizer for Quantum Error Correction Codes on Sparse Hardware Architectures Reviewer Persona: The Guardian

---

Summary

This paper presents QECC-Synth, a compilation framework to map Quantum Error Correction (QEC) circuits onto hardware with sparse qubit connectivity. The authors identify the limitations of existing swapping-based methods and manual bridging-based methods. Their proposed solution leverages the "ancilla bridge" technique and formalizes the mapping problem as a two-stage optimization process. The first stage, "Code Topology Mapping," uses a MaxSAT solver to determine data qubit placement and ancilla bridge allocation, optimizing for minimal ancilla overhead and maximal parallelism. The second stage, "Gate Scheduling," uses a SAT solver to find a valid, depth-minimized schedule for the resulting Error Syndrome Measurement (ESM) circuits. For large-scale codes, a heuristic partitioning scheme is proposed to maintain scalability. The authors evaluate their framework against swapping-based (Sabre, SATmap) and a specialized bridging-based (Surf-Stitch) method, claiming significant reductions in CNOT count and circuit depth across a variety of QEC codes and architectures.

Strengths

1. The formalization of the ancilla bridge design space as a constraint optimization problem is a clear and systematic contribution. Moving beyond ad-hoc heuristics and manual designs for bridging is a necessary step for the field. The breakdown into hard and soft constraints in Section 5.1 is logical.
2. The demonstration of the framework's ability to handle defective architectures (Table 2, page 11) is a significant practical advantage. Real-world hardware is imperfect, and methods that rigidly assume regular lattice structures (like Surf-Stitch) are inherently limited. QECC-Synth's flexibility here is a notable strength.
3. The scalability analysis presented in Table 3 (page 12) provides compelling evidence for the efficiency of the authors' MaxSAT encoding. Showing that their problem formulation results in dramatically fewer variables and clauses compared to a general-purpose mapping tool like SATmap effectively justifies their specialized approach.

Weaknesses

My primary concerns with this paper relate to a fundamental contradiction in its scaling strategy, unsubstantiated claims of optimality, and a superficial treatment of fault-tolerance implications.

1. Contradictory Heuristic for Scaling: The paper's core premise is that bridging methods are superior to swapping-based methods because they maintain fixed qubit locations, avoiding the overhead and complexity of routing (Section 1, page 2). However, the proposed solution for large-scale problems (Section 5.3, "Heuristic Approach," page 8) completely undermines this premise. The authors state, "SWAP gates are inserted between adjacent ESM circuits to reposition the qubits." This re-introduces the very routing problem the bridging method was meant to solve. The paper provides no quantification of this SWAP overhead. How many SWAPs are needed for the d=9 surface code? How does this CNOT and depth overhead compare to simply using a state-of-the-art SWAP-based router for the entire problem? Without this analysis, the heuristic approach appears to be a tacked-on fix that negates the paper's central argument.

2. Misleading Claims of Optimality: The authors repeatedly frame their approach as finding an optimal implementation. While SAT/MaxSAT solvers are exact solvers, this is only true if they run to completion. Table 1 (page 10) is replete with asterisks (*) indicating that the solver timed out and returned a sub-optimal result. For nearly every non-trivial combination of a complex code and a sparse architecture (e.g., HGP code on H-Square, [[81,1,9]] code on Hexagon), the "optimal" framework fails to find a proven optimal solution within the 7,200-second time limit. Therefore, for most practical purposes, the proposed tool is a heuristic that leverages a SAT solver. The claims of optimality should be significantly tempered to reflect this reality.

3. Arbitrary Optimization Objectives: In Section 5.1.2 (page 7), the authors define two soft constraints: minimizing total ancilla size (P1) and mitigating stabilizer conflicts (P2). They state they prioritize P1 over P2. This is a critical design decision that directly impacts the structure of the final circuit. However, it is presented without rigorous justification. It is conceivable that a solution with a slightly larger ancilla bridge that allows for complete parallel execution of stabilizer checks could result in a much shallower and ultimately higher-fidelity circuit. The paper lacks an ablation study or sensitivity analysis on the weights (w1, w2) of the objective function, making this key choice seem arbitrary.

4. Superficial Fault-Tolerance Analysis: The paper focuses almost exclusively on circuit metrics (CNOT count, depth) as proxies for performance. While these are important, the ancilla bridge itself introduces new potential fault paths. As seen in Figure 3(b), an error on an ancilla qubit within the bridge can propagate to multiple data qubits, creating a correlated error that may be difficult for the decoder to correct. The paper does not analyze the impact of different bridge structures on the spectrum of correlated errors. The claim of building a tool for "fault-tolerant quantum computing" (FTQC) requires more than just circuit optimization; it requires an analysis of how the synthesized circuits interact with a decoder under a realistic noise model.

Questions to Address In Rebuttal

1. Regarding the heuristic approach (Section 5.3): Please address the apparent contradiction of introducing SWAP gates in a framework designed to avoid them. Can you provide a quantitative analysis of the CNOT and depth overhead from these "integration layers" for the large-scale codes in Table 1? How does this total overhead compare against a state-of-the-art swapping-based approach like Sabre applied to the same problem?

2. Regarding claims of optimality: Given the frequent timeouts reported in Table 1, please justify the continued use of the word "optimal" to describe the solutions for large-scale, practical codes. Would it be more accurate to describe the framework as an exact solver for small instances and a SAT-based heuristic for larger ones?

3. Regarding the objective function (Section 5.1.2): Please provide a more rigorous justification for strictly prioritizing ancilla minimization (P1) over conflict mitigation (P2). Can you provide data from a small-scale experiment showing how the final circuit depth and CNOT count vary as the relative weights of these two objectives are changed?

4. Regarding fault tolerance: The ancilla bridge construction in Figure 3 introduces CNOTs that can propagate a single ancilla fault to multiple data qubits. Have the authors considered how such correlated errors affect the performance of the MWPM decoder used in the evaluation? Do certain bridge topologies generated by QECC-Synth produce error signatures that are more challenging for standard decoders than others?

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper introduces QECC-Synth, a novel, automated compiler for mapping Quantum Error Correction (QEC) circuits onto realistic quantum hardware with sparse connectivity. The core problem addressed is the significant architectural gap between the dense connectivity requirements of theoretical QEC codes (e.g., the surface code) and the limited, often irregular connectivity of physical quantum devices.

The authors' primary contribution is to move beyond the state-of-the-art in two ways. First, instead of using conventional swapping-based routing methods—which are ill-suited for QEC due to the need for static data qubit positions—they adopt the "ancilla bridge" technique. Second, and more importantly, they are the first to systematically formalize the design space of these ancilla bridges and create an automated synthesis framework to optimize their implementation. This is achieved through a two-stage compilation process that leverages MaxSAT and SAT solvers to (1) map the code topology (data qubits and ancilla bridges) and (2) schedule the resulting gate operations.

The evaluation demonstrates that QECC-Synth significantly outperforms both general-purpose, swapping-based compilers (Sabre, SATmap) and specialized, heuristic-based QEC compilers (Surf-Stitch). The framework shows superior results in terms of CNOT overhead and circuit depth, and crucially, exhibits broad applicability across diverse QEC codes and hardware architectures, including those with fabrication defects.

Strengths

1. Addresses a Critical Bottleneck in Fault-Tolerant Quantum Computing: The paper tackles one of a handful of truly critical, practical problems on the path to fault-tolerance. The gap between QEC code theory and hardware reality is a major source of overhead that could render many codes impractical. This work provides a concrete, powerful tool to bridge that gap, making it highly relevant and timely.

2. Systematization of a Promising Technique: The concept of ancilla bridging is not entirely new, but previous work has largely treated it with manual designs or code-specific heuristics. The key strength of this paper is the formalization of the ancilla bridge design space (as highlighted in Section 3 and Figure 4, page 4). By identifying flexibilities like bridge shape, size, and ancilla sharing, the authors transform an ad-hoc technique into a well-defined optimization problem. This is a significant step in maturing the field of quantum compilation from an art into a science.

3. Excellent Generality and Practicality: A standout feature of QECC-Synth is its generality. The framework is not hardcoded for a specific code or architecture. The evaluation convincingly shows its effectiveness on surface codes, color codes, and even modern qLDPC codes. Furthermore, its ability to handle defective hardware (Table 2, page 11) is a massive practical advantage. Real-world quantum processors have defects, and a compiler that can gracefully work around them is far more useful than one that assumes a perfect, regular grid. This significantly broadens the potential impact of the work.

4. Strong and Comprehensive Evaluation: The authors conduct a rigorous evaluation against the right set of baselines. Comparing against both swapping-based methods (Sabre, SATmap) and a specialized bridging method (Surf-Stitch) clearly situates the contribution. The results in Table 1 (page 10) are compelling, showing not just quantitative improvements but also the ability to find solutions where other methods fail entirely ("Time-Limit" or "Not Exist"). Connecting the compiler-level metrics (CNOT count, depth) to the ultimate physical metric (logical error rate, as shown in Figure 9, page 11) closes the loop and validates that the optimizations truly matter for fault tolerance.

Weaknesses

While the work is strong, there are areas where its conceptual boundaries and practical trade-offs could be further explored.

1. Scalability Heuristic is Under-detailed: The optimal SAT-based approach is naturally limited in scale. The authors acknowledge this and propose a partitioning-based heuristic for large codes (Section 5.3, page 8). While this is a sensible strategy, the description is quite high-level. The process of "sorting the SAT problems" to minimize routing between partitions seems crucial to the quality of the final result, yet the details are sparse. This makes it difficult to assess how much optimality is sacrificed for scalability and how sensitive the results for large codes (like the d=9 surface code) are to this heuristic.

2. Parameterization of the Objective Function: The MaxSAT formulation for the topology mapping (Stage 1) relies on a weighted objective function to balance minimizing ancilla size against maximizing stabilizer compatibility (Section 5.1.2, page 7). The choice of weights (w1, w2) can significantly influence the solution. The paper does not discuss how these weights were chosen or the sensitivity of the results to them. It would be valuable to understand the trade-offs at play here—is there a Pareto frontier of solutions that users might want to explore?

Questions to Address In Rebuttal

1. Regarding the partitioning heuristic for large-scale problems (Section 5.3), could the authors provide more insight into the "sorting" process for the sub-problems? For instance, is the selection of the next subset S_j based purely on the number of overlapping data qubits, or are other metrics considered? What is the typical SWAP overhead incurred during the "Integrating the ESM circuits" step, and how does this compare to the CNOTs saved by the core algorithm within each partition?

2. Could you elaborate on the process for selecting the weights (w1, w2) in the Stage 1 MaxSAT objective function? Have you explored the trade-off space between minimizing total ancilla qubits (which reduces CNOTs within each ESM block) and minimizing stabilizer conflicts (which increases parallelism and reduces depth)? Presenting even a small example of this trade-off would strengthen the paper.

3. Looking forward, how extensible is the QECC-Synth framework? It is currently built around the ancilla bridge technique. Could the constraint-based model be adapted to incorporate other hardware-specific features for mitigating connectivity issues, such as long-range couplers or shuttle-based qubit movement, or is it fundamentally tied to the local, bridging-based model of interaction?

Reviewer Persona: The Innovator (Novelty Specialist)

---

Summary

The authors present QECC-Synth, a compilation framework designed to map Quantum Error Correction (QEC) circuits, specifically Error Syndrome Measurement (ESM) circuits, onto hardware with sparse qubit connectivity. The core problem—the mismatch between the connectivity requirements of QEC codes and the physical limitations of hardware—is well-established. The authors propose to solve this using an "ancilla bridge" technique.

The central claim to novelty in this paper is not the ancilla bridge technique itself, but rather a new, automated methodology for synthesizing these bridges. The authors assert that prior work has overlooked the vast design space of these bridges. Their contribution is twofold: (1) a systematic classification of the flexibilities within this design space (e.g., bridge shape, size, ancilla sharing), and (2) the formalization of the synthesis problem as a two-stage MaxSAT/SAT optimization problem that can automatically explore this space to find optimal mappings. The first stage handles the "Code Topology Mapping" (qubit placement and bridge allocation), while the second stage tackles "Gate Scheduling."

---

Strengths

1. Novel Formalization of a Known Technique: The most significant contribution of this work is the successful transition of the "ancilla bridge" concept from a subject of theoretical exploration and manual, bespoke design into a domain of automated, formal synthesis. Prior works, such as Lao and Almudever [36] and Chamberland et al. [14], introduced and manually applied bridging/flag concepts for specific codes. This paper's novelty lies in abstracting the principles of bridging into a set of formal constraints and objectives that can be solved by a general-purpose engine (MaxSAT). This represents a clear and significant step beyond the prior art.

2. Systematic Identification of a New Design Space: The paper correctly identifies that previous applications of bridging were essentially point solutions. The authors' classification of bridge flexibilities in Section 3—covering variations in shape and size, and the possibility of shared/overlapped ancillas—is a novel conceptual contribution. By identifying and categorizing these degrees of freedom, they have defined a new, richer optimization problem that was not explicitly addressed before.

3. Clear Differentiation from Swapping-Based Synthesis: The authors successfully argue that existing automated synthesis methods, which are predominantly swapping-based (e.g., SATmap [42]), are fundamentally ill-suited for QEC due to the requirement of fixed data qubit positions. By creating a formal framework specifically for a non-swapping technique, they have addressed a clear gap in the compiler toolchain for fault-tolerant quantum computing.

---

Weaknesses

While the application of the framework is novel, one could argue that the core components are established techniques.

1. Incremental Novelty of the Methodological Components: The use of MaxSAT for mapping and scheduling problems in quantum compilation is not new in itself, with SATmap [42] being a prominent example. Similarly, decomposing a complex synthesis problem into stages (e.g., placement then scheduling) is a standard practice in the EDA community. Therefore, the novelty is not in the choice of a MaxSAT solver or a two-stage approach, but strictly in the specific encoding and problem definition for the ancilla bridge design space. The authors should be careful to frame their contribution this way, as the methodology itself is an application of known computer science techniques.

2. Clarity of the "Delta" from Heuristic Bridging: The paper compares against Surf-Stitch [64], a heuristic bridging-based method. While the results clearly show QECC-Synth is more general and often superior, the conceptual novelty could be sharpened. Surf-Stitch also automates a form of bridging, albeit heuristically and for a specific code family. The key delta is the move from a specialized heuristic to a general, formal, and optimal (within the SAT solver's limits) method. This distinction is crucial and could be stated more forcefully as the primary intellectual contribution.

---

Questions to Address In Rebuttal

To solidify the paper's claims of novelty, I would ask the authors to address the following:

1. Please explicitly delineate the conceptual boundary between this work and prior art on ancilla bridges (e.g., [36], [14]). Is it fair to say that the prior art established the physical mechanism of bridging, while your work provides the first algorithmic framework for automatically synthesizing them on a general basis?

2. The two-stage SAT formulation (Topology Mapping followed by Gate Scheduling) is a logical decomposition. To what extent is this decomposition novel versus a standard approach for such synthesis problems? Is there a unique insight in this particular decomposition that is specific to the QEC bridging problem and enables tractability?

3. Your systematic classification of the design space in Section 3 is a cornerstone of your contribution. Are there any other potential flexibilities in the ancilla bridge design space that your current formalism does not capture? Discussing the limitations of your novel abstraction would strengthen the paper. For instance, does your model account for heterogeneous hardware where ancilla qubits might have different coherence times or connectivity, influencing the optimal bridge shape?

Paper Title: RANGE-BLOCKS: A Synchronization Facility for Domain-Specific Architectures Reviewer: The Guardian

---

Summary

This paper proposes Range-Blocks (RBlox), a hardware synchronization facility designed to support dynamic data structures on domain-specific architectures (DSAs). The authors argue that existing DSAs are limited to static data structures due to the lack of efficient synchronization, forcing them into inefficient batch updates on a host CPU or costly address-based atomics. The core idea is to replace address-based locks with locks on key ranges, which better reflect the logical organization of hierarchical data structures. The proposed hardware consists of two main components: an LTable to track active locks and a UTable to cache recently unlocked "safe" ranges for fast re-acquisition. The authors evaluate their design, RBlox++, on a 128-tile dataflow DSA simulator across several workloads, claiming significant improvements in performance (up to 15x), DRAM bandwidth, and energy compared to baselines including a host-based batching model, a hardware lock cache (LCache), and an in-memory range list (R-List).

Strengths

1. Well-Motivated Problem: The paper correctly identifies a critical limitation in current DSAs—the inability to efficiently handle dynamic data structures. This is a relevant and important problem for expanding the applicability of DSAs.

2. Plausible Core Idea: The central concept of using logical key ranges for synchronization instead of physical memory addresses is sound. It decouples synchronization from the memory layout and has the potential to reduce the number of lock operations required for mutations in hierarchical structures like B+trees, as demonstrated in Figure 1.

3. Comprehensive Evaluation Suite: The authors evaluate their proposal against a reasonable set of benchmarks (Database Scan, PageRank, KV-Store, etc.) that represent the target application domain. The inclusion of multiple competing designs, particularly the LCache and R-List, provides a basis for comparison, although the fairness of these baselines is debatable.

Weaknesses

My primary concerns with this manuscript revolve around overstated claims, questionable baseline comparisons, and a lack of rigor in defining the general applicability of the proposed mechanism.

1. Misleading Hardware Cost Comparison: The abstract makes the striking claim that RBlox requires a small 2kb on-chip table compared to 256kb for address-based locks. This is disingenuous. The 2kb figure appears to refer only to the LTable for a 128-tile configuration (128 entries). The performance benefits of the full RBlox++ system, however, critically depend on the much larger UTable. In the evaluated configuration (Table 2, page 9), the UTable has 4k entries. Assuming a similar entry size to the LTable, this constitutes a significant storage cost (likely >80KB), not 2kb. The 256kb figure for LCache is also presented without sufficient context—it is simply the size chosen for the evaluation, not an inherent requirement of the address-locking paradigm. This framing creates a false narrative of overwhelming hardware efficiency.

2. Overstated "Instant" Locking Claim: The paper repeatedly uses the term "instant" to describe the UTable-based lock acquisition in RBlox++ (e.g., "instantly achieve mutual exclusion" in the abstract). This is technically inaccurate. The mechanism is a cache lookup (Figure 5), which incurs latency (specified as 5 cycles/bank in Table 2). More importantly, it is contingent on a hit in the UTable. Its effectiveness is probabilistic, not deterministic or "instant." The data in Section 7.2 suggests lock elision rates of 30-50%, implying that this "instant" path is unavailable half the time or more. The terminology should be corrected to "optimistic" or "accelerated" to reflect the true nature of the mechanism.

3. Weakness of the Primary Baseline ("Base"): The headline 15x speedup claim is derived from a comparison against the "Base" hybrid model, where a host multicore performs batched updates while DSA readers are stalled. This is an exceptionally weak, non-scalable strawman. Any fine-grained, tightly-coupled hardware accelerator is expected to outperform such a loosely-coupled, coarse-grained software approach. The technically meaningful comparisons are against LCache and R-List, where the speedups, while still notable, are a more modest 6.8x and 12.5x, respectively. The abstract and conclusions should be re-framed to emphasize these more honest comparisons.

4. Unclear Generalizability and Ambiguity in Range Definition: The B+tree is used as the canonical example throughout the paper, as its hierarchical key-space maps cleanly to the [Lo, Hi] range concept. The application to other data structures is not convincingly argued. For instance:

   • Hash Tables: The paper dismisses the challenge of non-integer keys by stating "We hash all key types to an integer key" (Section 4, page 7). This is insufficient. The critical challenge in a dynamic hash table is a resize operation, which re-maps a large number of keys to new buckets. It is entirely unclear how a "range lock" would work in this context, as the relationship between key ranges and the physical data structure (buckets) is non-local and globally redefined.
   • Graphs: The PageRank example (Section 4.1) defines a range as [u.VLo, u.VHi], concatenating the source vertex ID to disambiguate. This feels like an ad-hoc solution. What happens when a vertex is deleted, or when a super-vertex is split? The semantics of "range" become fragile when the underlying key space is not as rigidly ordered as in a B+tree.

Questions to Address In Rebuttal

1. Please justify the 2kb vs. 256kb hardware cost comparison in the abstract. A rigorous rebuttal must include the full area/storage cost of the complete RBlox++ mechanism (LTable + UTable) used to achieve the reported performance, and explain why the selected 256KB LCache is the appropriate point of comparison.

2. Can you defend the use of the term "instant" for the RBlox++ fast-path locking? Please provide UTable hit-rate data across all evaluated benchmarks to quantify the actual applicability of this fast path.

3. Given that the "Base" model is a weak baseline, please re-frame your primary performance claims relative to the LCache implementation. Why should the community accept a 15x claim when the speedup over a more comparable hardware baseline is 6.8x?

4. Provide a detailed, step-by-step example of how Range-Blocks would handle a full resize and re-hashing operation in a dynamic hash table. Your explanation must clarify how [Lo, Hi] ranges are defined, locked, and released during this global data structure transformation.

5. The energy for an RBlox access is listed as 3750 fJ per bank (Section 6). This seems high for a simple SRAM array access. Please detail the methodology used to arrive at this figure and break down the components of this energy cost.

Paper Title: RANGE-BLOCKS: A Synchronization Facility for Domain-Specific Architectures Reviewer Persona: The Synthesizer (Contextual Analyst)

---

Summary

This paper addresses a crucial and timely problem: the lack of support for dynamic, mutable data structures on domain-specific architectures (DSAs). The authors correctly observe that current DSAs excel at static, affine workloads but falter when data needs to be inserted or deleted, forcing developers into inefficient solutions like batching updates on a host CPU.

The core contribution is Range-Blocks (RBlox), a hardware synchronization facility for DSAs. The central idea is to shift from traditional address-based locks to symbolic, key-range-based locks. This insight is powerful because the logical key ranges of a data structure (e.g., the keys covered by a B+tree node) become the primitive for synchronization. The authors propose a compact, on-chip hardware implementation consisting of two tables: an LTable to track active locks and a UTable to cache recently unlocked, "safe" ranges for fast re-acquisition. This design enables significant performance improvements by reducing the number of synchronization operations, minimizing off-chip traffic, and enabling "instant locking" on known-safe sub-trees, bypassing expensive top-down traversals.

---

Strengths

1. Excellent Problem Identification and Motivation: The paper identifies a fundamental weakness in the current trajectory of DSA design. As accelerators move beyond scientific computing and into data-centric applications like graph analytics, databases, and KV-stores, the need for efficient dynamic updates becomes paramount. This work tackles a real, significant, and forward-looking problem.

2. Elegant Core Idea: The central concept of using key ranges as the locking primitive is a beautiful synthesis of data structure theory and hardware architecture. While address-based locks are oblivious to the data structure's semantics, range locks are intrinsically aware of its hierarchical organization. This semantic coupling allows for far more efficient synchronization protocols, as demonstrated by the ability to lock an entire sub-tree with a single operation. This is a classic example of hardware-software co-design done right.

3. Strong Grounding in Concurrent Data Structure Theory: The RBlox++ design, with its UTable, is a clever hardware acceleration of a known software optimization in concurrent B+trees: identifying "safe" nodes where modifications are guaranteed not to propagate upwards. By caching these safe ranges, the hardware allows future operations to bypass the traversal from the root, directly acquiring a lock on the narrowest possible region. This demonstrates a deep understanding of the algorithms the hardware is meant to accelerate.

4. Establishes a New Design Point: This work proposes a new architectural primitive that could fundamentally alter how we design DSAs for dynamic applications. It moves beyond simply porting multicore CPU concepts (like the hardware lock cache, LCache, which they show is less effective) and instead creates a solution tailored to the needs of data structures and the constraints of DSAs. It provides a blueprint for what "synchronization support" could look like in this domain.

---

Weaknesses

While the core idea is strong, the paper could be strengthened by better contextualizing its contribution and exploring the boundaries of its applicability.

1. Framing of Novelty: The paper presents range-based locking as a novel concept. However, key-range locking is a well-established and foundational technique in the database community, dating back decades (e.g., work by Mohan on ARIES/KVL, and predicate locking in general). The true novelty of this paper is not the concept of range locking, but its efficient, specialized hardware implementation for DSAs and its co-design with the dataflow execution model. The paper would be much stronger if it explicitly acknowledged this lineage and framed its contribution as adapting and accelerating a proven database concept for a new architectural paradigm.

2. Assumed Generality: The primary examples and evaluation are centered on ordered, tree-like data structures (B+trees, skip lists) where key ranges are a natural concept. It is less clear how well the Range-Blocks primitive would apply to other important dynamic structures. For example, in a dynamically resizing hash table, what constitutes a "range"? While the authors suggest hashing keys to an integer space (Section 4, page 7), a hash function's goal is to distribute keys pseudo-randomly, making contiguous integer ranges less meaningful. An update causing a table resize would invalidate many existing ranges simultaneously. The paper would benefit from a discussion of these limitations or a more detailed proposal for handling such cases.

3. Software/Compiler Complexity is Understated: The paper presents a clean API, but glosses over the significant challenge of how the software or a compiler would extract the [Lo, Hi] bounds from data structure nodes and manage the logic for trimming and upgrading locks. This hardware-software contract is critical to making the system work. A brief discussion on the compiler passes or programming model idioms required to effectively use RBlox would add significant practical weight to the proposal.

---

Questions to Address In Rebuttal

1. Could the authors please clarify the novelty of their work with respect to the long history of key-range locking in database systems? Positioning this work as a novel hardware architecture to accelerate these established software concepts would, in my view, strengthen the paper's contribution.

2. Beyond the B+tree and skip-list examples, how would the Range-Blocks facility handle the synchronization of a dynamic hash table, particularly during a table-wide resize operation where many disjoint keys must be moved and coordinated?

3. What is the expected complexity on the software/compiler side to use RBlox? Who is responsible for identifying the key ranges within nodes during traversal, and is this something that can be automated for common data structures or does it require significant manual programmer effort?