Overview
This research investigates finite-particle convergence rates for both conservative and non-conservative drifting methods applied to one-step generative modeling. A conservative drifting method is proposed, which replaces the original displacement-based drifting velocity with a kernel density estimator (KDE)-gradient velocity. This velocity is derived from the difference between the kernel-smoothed data score and the kernel-smoothed model score. The study also examines a previously proposed non-conservative drifting method that utilizes a Laplace kernel.
Research Context
The core problem addressed is the non-conservatism identified in general displacement-based drifting fields within generative modeling. The proposed conservative method aims to address this issue by defining a velocity field as a gradient field. The non-conservative method investigated in this work corresponds to the original displacement-based velocity introduced in Deng et al., 2026 (arxiv:2602.04770).
Approach
For the conservative drifting method, the velocity is formulated as a KDE-gradient velocity. This involves computing the difference between a kernel-smoothed data score and a kernel-smoothed model score. The analysis of this method on $\R^d$ involves proving continuous-time finite-particle convergence bounds. These bounds are derived using a joint-entropy identity, yielding results for the empirical Stein drift, the smoothed Fisher discrepancy of the KDE, and the squared center velocity.
Key aspects of the analysis for the conservative method include:
- The main finite-particle correction term identified is a reciprocal-KDE self-interaction term.
- Deterministic and high-probability local-occupancy conditions are provided to control this self-interaction term.
- Quadrature constants are explicitly tracked, including their potential dependence on bandwidth ($h$).
- Specific root residual-velocity rates are identified:
- A rate of $N^{-1/(d+4)}$ is observed under an additional $h$-uniform quadrature regularity condition.
- A more general optimized root rate of $N^{-(2-\beta)/(2(d+4-\beta))}$ is found under a growth condition, where $0 \le \beta < 2$.
For the non-conservative drifting method, which uses a Laplace kernel, the approach involves a decomposition of the velocity. A sharp companion kernel is utilized to decompose the velocity into a positive scalar preconditioning of a sharp-score mismatch, plus a Laplace scale-mismatch residual. This decomposition facilitates the production of an analogous finite-particle rate, which includes an unavoidable residual term.
Findings
- The proposed conservative drifting method utilizes a KDE-gradient velocity, which functions as a gradient field, thereby addressing non-conservatism in displacement-based drifting fields.
- Continuous-time finite-particle convergence bounds were established for the conservative method on $\R^d$. These bounds specifically quantify the empirical Stein drift, the smoothed Fisher discrepancy of the KDE, and the squared center velocity.
- A reciprocal-KDE self-interaction term was identified as the primary finite-particle correction for the conservative method. Control of this term is achieved via specified deterministic and high-probability local-occupancy conditions.
- The root residual-velocity rate for the conservative method is $N^{-1/(d+4)}$ under an $h$-uniform quadrature regularity condition. A more general growth condition yields an optimized root rate of $N^{-(2-\beta)/(2(d+4-\beta))}$ (where $0 \le \beta < 2$).
- For the non-conservative drifting method with a Laplace kernel, the velocity can be decomposed using a sharp companion kernel into a positive scalar preconditioning of a sharp-score mismatch and a Laplace scale-mismatch residual.
- The non-conservative method exhibits an analogous finite-particle rate, which includes an unavoidable residual term.
- The continuous-time residual-velocity bounds translate into one-step generation guarantees, facilitated by the explicit drift size $\eta$.