Overview
This study introduces two new approaches for training multi-column radial basis function neural networks (RBFNs): Multi-Column RBFN with Particle Swarm Optimization (MC-PSO) and Multi-Column RBFN with Adaptive Particle Swarm Optimization (MC-APSO). These methods integrate evolutionary swarm algorithms into a parallel RBFN structure to address scalability challenges encountered by existing RBFN training techniques, particularly with large datasets.
Research Context
Radial basis function neural networks (RBFNs), when trained using gradient descending algorithms, offer an effective fully connected structure for both shallow and deep networks. The error correction (ErrCor) method, a gradient-based training approach, selects optimal hidden units to enhance accuracy. Conversely, particle swarm optimization (PSO), a population-based algorithm, leverages swarm experience to optimize RBFN parameters, providing global search capabilities and robustness against local minima. Adaptive PSO (APSO) is an enhanced variant of PSO designed to improve convergence speed through dynamic adjustment of swarm parameters during optimization. Both ErrCor and PSO have demonstrated improved results and competitive convergence characteristics.
However, the application of these methods to large datasets presents scalability issues, specifically related to extensive kernel computations and the architecture of large hidden layers. A prior approach, the multi-column RBFN (MCRN), aimed to improve ErrCor performance by deploying multiple small RBFNs in a parallel system. The current research is inspired by the MCRN's success in mitigating scalability challenges.
Approach
Inspired by the MCRN's architecture, the study proposes MC-PSO and MC-APSO. These methods involve implementing parallel RBFN structures, which are then trained using evolutionary swarm-based algorithms. Within this framework, each individual RBFN is independently trained on a specific spatial subset of the entire dataset. The training is conducted using either the PSO algorithm for MC-PSO or the APSO algorithm for MC-APSO. This specialized training process results in RBFNs that are tailored to their respective data subsets.
During the testing phase, the multi-column output is generated by contributions solely from selected RBFNs. These contributing RBFNs are those whose trained specializations align with the neighborhood of the particular test instance. This mechanism aims to leverage the specialized knowledge of each RBFN column while benefiting from the parallel architecture to enhance processing speed.
Findings
- The proposed MC-PSO and MC-APSO methods demonstrated superior performance in terms of accuracy when compared to ErrCor, PSO, APSO, and MCRN.
- Both MC-PSO and MC-APSO exhibited higher recall values relative to ErrCor, PSO, APSO, and MCRN.
- In most experiments conducted, MC-PSO and MC-APSO achieved faster training times than the comparative methods.
- The testing times for MC-PSO and MC-APSO were also found to be faster in the majority of experiments when compared to ErrCor, PSO, APSO, and MCRN.
Why This Matters
The observed improvements in accuracy, recall, and computational speed for MC-PSO and MC-APSO suggest these methods could offer more efficient and effective solutions for RBFN training, especially when dealing with large datasets where existing methods face scalability limitations such as extensive kernel computations and large hidden layer structures.