Overview
Kernelized graph methods, encompassing techniques like spectral clustering, diffusion maps, and sparse kernel regression graphs, rely on Gaussian kernels. A critical parameter in these methods is the Gaussian bandwidth, denoted as $\sigma$. This parameter directly influences the spectral characteristics of the local kernel operator. The selection of $\sigma$ is crucial, as an excessively small value leads the kernel to overstate local complexity, treating each sample as an independent directional entity. Conversely, an overly large $\sigma$ can cause the kernel to consolidate multiple directions, resulting in a divergent condition number and a loss of geometric discrimination.
To address this challenge, a novel approach is introduced to determine a scale that aligns the spectral complexity of the kernel with the intrinsic complexity of the underlying manifold. This involves a per-node bandwidth criterion designed to operationalize this principle. The criterion functions by jointly equating the kernel's effective rank with the local intrinsic dimension, which is estimated through a minimum spanning tree technique. This process is anchored within the manifold-consistent log-log scaling regime.
Research Context
The efficacy of kernelized graph methods, such as spectral clustering, diffusion maps, and sparse kernel regression graphs, is fundamentally linked to the choice of the Gaussian bandwidth $\sigma$. This parameter dictates the spectral properties of the local kernel operator. Suboptimal choices for $\sigma$ can compromise the performance and interpretability of these methods. Specifically, a $\sigma$ value that is too small can lead to an overestimation of local complexity, causing each sample to be perceived as an independent direction. Conversely, a $\sigma$ that is too large can result in the collapse of distinct directions, leading to a diverging condition number and the loss of the ability to discriminate geometric features.
Approach
The proposed methodology focuses on establishing a scale that ensures the spectral complexity of the kernel aligns with the intrinsic complexity inherent to the underlying manifold. Central to this approach is a per-node bandwidth criterion. This criterion is designed to match the kernel's effective rank with the estimated local intrinsic dimension. The local intrinsic dimension is determined using a minimum spanning tree. The entire search for the appropriate bandwidth is grounded in the manifold-consistent log-log scaling regime.
Findings
Evaluation of the proposed adaptive bandwidth method utilized SSL (self-supervised learning) embeddings derived from six distinct encoders. These embeddings were applied to the CIFAR-100 dataset. The results indicated that the adaptive bandwidth strategy consistently improved accuracy in both leave-one-out (LOO) classification and label propagation (LP) tasks. This improvement was observed when compared against methods employing fixed-bandwidth settings, as well as against other existing adaptive methods.
Why This Matters
The effective selection of the Gaussian bandwidth is a critical factor in the performance and reliability of kernelized graph methods. The development of an adaptive, geometry-aware approach to bandwidth control offers a mechanism to consistently enhance the accuracy of tasks like leave-one-out classification and label propagation. This is particularly relevant in contexts utilizing SSL embeddings and datasets like CIFAR-100, where precise geometric discrimination is essential.