Overview
This study investigates low-precision computation of matrix product $C=AB$ where both matrices $A$ and $B$ are quantized. A central contribution is the derivation of an exact finite-dimensional identity for the expected squared product error. This identity is applicable under specific conditions, namely independent, zero-mean entrywise errors with known variance fields. The derived identity holds precisely for non-overloading subtractive dither and for independent stochastic rounding. The researchers also empirically assessed deterministic round-to-nearest (RTN) quantization.
A primary methodology introduced is 'contraction-gauge preconditioning'. This technique leverages the product-preserving equivalence $AB=(AT)(T^{-1}B)$ to jointly select a factor representation and its sharing pattern prior to quantization. The objective of this preconditioning is to reduce product error, though its implementation may necessitate additional transformed and quantized copies of the opposite operand.
Research Context
The core problem addressed is the accurate computation of matrix products $C=AB$ in low-precision environments where both input matrices $A$ and $B$ are quantized. Quantization introduces errors, and understanding and mitigating these errors is crucial for the deployment of low-precision systems. The research contributes to the field by providing a rigorous mathematical framework for error analysis and a practical preconditioning method.
Approach
The research began by deriving an exact finite-dimensional identity for the expected squared product error. This identity was established for independent, zero-mean entrywise errors with known variance fields. It applies to non-overloading subtractive dither and independent stochastic rounding. For deterministic round-to-nearest (RTN), its performance was empirically assessed.
The new method, contraction-gauge preconditioning, is based on the identity $AB=(AT)(T^{-1}B)$. This method involves jointly selecting a factor representation and its sharing pattern before quantization. The preconditioning strategy aims to reduce product error. Depending on the transform's sharing pattern, it may require either one copy (for a shared transform) or up to one copy per block (for a block-specific transform) of the transformed, quantized opposite operand.
Within the family of positive diagonal gauges, specifically termed 'folds', a geometric program (GP) was employed to compute a globally optimal shared fold. For determining if the identity fold is already optimal, a linear program was utilized. For other families of transforms, the researchers derived computable selection statistics. These statistics include the tail index for scaling, profile spread for partitioning, coherence and weighted-Gram energy for rotations, and slice-energy covariance for hierarchy depth. These statistics facilitate the ranking of heuristic candidate transforms through upper bounds.
Findings
- An exact finite-dimensional identity for the expected squared product error was derived for $C=AB$ with quantized factors, valid for independent, zero-mean entrywise errors with known variance fields.
- This identity holds exactly for non-overloading subtractive dither and for independent stochastic rounding.
- Contraction-gauge preconditioning, based on $AB=(AT)(T^{-1}B)$, involves jointly choosing a factor representation and its sharing pattern before quantization to reduce product error.
- Optimal shared folds within the positive diagonal gauge family can be computed via a geometric program, and a linear program can determine if the identity fold is optimal.
- For other transform families, computable selection statistics (tail index, profile spread, coherence, weighted-Gram energy, slice-energy covariance) were derived, providing upper bounds for ranking candidates.
- Across twelve linear products from a trained three-block image classifier, median within-product rank correlations between dither-model predictions and deterministic-RTN errors were 0.937 at 8 bits and 0.918 at 4 bits.
- The GP-optimized fold reduced held-out product error over the identity fold by 18.0% (8-bit) and 20.5% (4-bit) in geometric mean.
- The GP-optimized fold outperformed a SmoothQuant-style grid baseline at both 8-bit and 4-bit precisions, and on ten of the twelve products examined.
- The GP-optimized fold lowered composed logit Mean Squared Error (MSE) by 15.4% and 26.4%.
Why This Matters
This research provides a precise method for accounting for stochastic product-error in quantized matrix multiplication. It offers certified selection mechanisms for optimal transforms within specific families and establishes a common objective for evaluating reusable transform candidates under RTN. The demonstrated error reductions in image classifier products suggest practical benefits for low-precision deep learning hardware and applications where computational efficiency and accuracy are critical.