Overview
Diffusion models are utilized for generating high-quality images and videos. The iterative denoising process inherent in these models can be computationally intensive. Training-free accelerators have emerged to mitigate this computational cost, often by reusing cached intermediate features or forecasting future ones. These methods sometimes incorporate an exact block feature computation for verification purposes, typically to measure discrepancies or guide decisions, after which the feature is discarded. FeatFix introduces a mechanism to reuse this previously computed exact feature for correction, specifically at fixed sparse layer–timestep sites. By forwarding the exact feature at the verification site, FeatFix aims to reset the local draft residual and reduce downstream feature error, contributing to faster cached diffusion inference.
Research Context
The computational intensity of iterative denoising in diffusion models motivates the development of acceleration techniques. A class of training-free accelerators addresses this by leveraging cached intermediate features or predicting future features. Within this class, some methods compute an exact block feature. This exact feature's primary role is often for verification, specifically to quantify discrepancy or inform subsequent decisions, before being discarded. The observed underutilization of this computed exact feature forms the basis for FeatFix, which posits that this feature can be repurposed for corrective actions within the inference process.
Approach
FeatFix is designed as a local exact-feature correction method. It identifies that the exact block feature, computed during verification, can be reused for correction rather than being discarded. The core mechanism involves forwarding this exact feature at the verification site. This action aims to reset the local draft residual, consequently reducing errors in downstream features. FeatFix operates by applying this correction at a predefined, sparse set of layer–timestep sites. At these selected sites, the method replaces the complete draft block output with the exact output that was computed from the same incoming state. This replacement strategy avoids partial token- or channel-level alterations and also bypasses the need for full-timestep recomputation. The intervention is localized to specific sites, integrating the exact feature reuse for enhanced efficiency.
Findings
Experiments were conducted across four distinct image and video backbones to evaluate FeatFix. The method consistently demonstrated acceleration in generation processes. FeatFix achieved a speedup of up to $6.70\times$ when compared to a 'Vanilla' approach. This acceleration was observed while maintaining a competitive level of output quality.