Offline — Build experience
Retain removals that preserve correctness and increase latency beyond profiling noise.
Ascend NPU, training-free optimization
AscendOptimizer
Episodic Agent for Ascend NPU Operator Optimization
Turning strong kernels into reusable experience — and hardware feedback into faster operators.
1.21×Overall GM speedup
101AscendC operators
1.89×Level 3 GM speedup
Method
Rewind. Retrieve.
Refine.
Starting from correct AscendC implementations, the agent removes optimization motifs, validates slowdowns, and distills inverse edits into reusable experience.
System overview. Rewind builds an experience bank; kernel refinement and tiling search use real-device feedback. Enlarge ↗
Stage I — Refine kernels
Retrieve from the frozen bank; retain kernel rewrites that pass correctness checks and reduce measured latency.
Stage II — Search tiling
Keep the refined kernel fixed; evolve tile sizes, core counts, and resource policies on hardware.
Results
Measured on real hardware.
Ascend 910B2 · CANN 8.3 · DeepSeek-V3.2
Geometric-mean speedup over cann-ops references
Results and figure numbering follow the revised manuscript; they differ in places from arXiv v2.
Geometric-mean speedup · 101 operatorsHigher is better ↑
Speedup over cann-ops; common 0–2× scale. Levels group engineering complexity. CudaForge is adapted to AscendC. Tables 2–3.
Same budget. Stronger results. Main-comparison methods receive 230 compile–correctness–profiling attempts per operator, including failures and timeouts. AscendOptimizer uses 20 for Rewind and 210 for online optimization.
Transductive main benchmark. The experience bank is built across the same 101 operators. Held-out transfer is evaluated separately.
Component ablations
Independent runs: tiling only 1.02× GM; kernel refinement without experience 1.09×; kernel refinement with Rewind 1.14×; full system 1.21×. These are not sequential checkpoints (Table 3).
Rewind vs. documentation
On a matched 25-operator subset, Rewind reaches 1.24× GM versus 1.12× for a documentation-derived bank, with the retriever, prompts, harness, and budget held fixed (Table 4).
Transfer to eight held-out operators
On eight held-out loss operators with DeepSeek-V4-Pro and equal budgets, the frozen-bank system reaches 1.06× GM versus 1.02× with Rewind and retrieval disabled. Neither these operators nor their results enter the bank (§4.6).
Evidence in practice
Reusable knowledge. Complementary gains.
Original paper figures · Click either image for full resolution
Cross-operator transfer. A loop rewrite reduces upsample_nearest_exact3d latency from 156 μs to 145 μs. With the same oracle bottleneck, each no-retrieval condition (with or without official documentation) reaches 145 μs in 1/100 attempts (§4.4). Enlarge ↗
Kernel + tiling optimization. On matmul_all_reduce, Stage I reaches 1.35×; Stage II raises speedup to 2.00× over the original reference (Appendix C). Enlarge ↗