ICANEWS

Engineering & Technology

Energy Bounds of Analog Compute-in-Memory with Local Normalization for Edge AI Workloads

This research investigates energy bounds within analog Compute-in-Memory (CIM) architectures, particularly concerning the integration of local normalization. Th

Overview

This research investigates energy bounds within analog Compute-in-Memory (CIM) architectures, particularly concerning the integration of local normalization. The work addresses challenges in achieving maximum energy efficiency for modern edge AI workloads, specifically those involving Large-Language-Models (LLMs) and low-bit floating-point formats.

Research Context

Modern edge AI workloads necessitate maximizing energy efficiency in computational hardware. Analog CIM architectures are being explored to meet this demand. Concurrently, the increasing prevalence of LLMs has driven the adoption of low-bit floating-point formats, which prioritize dynamic range. Conventional direct-accumulation CIM designs accommodate floating-point numbers by normalizing them to a shared, widened fixed-point scale. This normalization strategy results in hardware resolution being determined by the input's dynamic range rather than its intrinsic precision. Consequently, energy consumption in such systems is predominantly attributed to the Analog-to-Digital Converter (ADC).

Approach

The research proposes a method to address the limitation of conventional direct-accumulation CIM by introducing local normalization for each input, weight, and multiply-accumulate (MAC) output. This is achieved through a specific architectural component termed the Gain-Ranging MAC (GR-MAC). The overhead associated with this local normalization process is handled by low-power digital logic. This design strategy allows the computationally intensive MAC operation to remain within the energy-efficient, low-precision analog domain.

Energy modeling was employed to evaluate the proposed GR-MAC architecture. The modeling analyzed the impact of adding a gain-ranging stage to the MAC unit, specifically focusing on its effect on input dynamic range and energy consumption. The study considered a standard Signal-to-Quantization Noise Ratio (SQNR) of 35 dB.

Findings

  • The addition of a gain-ranging stage to the MAC unit enables a 4-bit increase in input dynamic range without incurring increased energy consumption, evaluated at a 35 dB SQNR standard.
  • The proposed local normalization makes the ADC resolution requirement invariant to assumptions about the input distribution.
  • This invariance allows for the construction of an upper bound with a 1.5-bit reduction in ADC resolution requirements compared to the conventional lower bound.

Why This Matters

These results establish a pathway toward unlocking favorable energy scaling trends for analog CIM in the context of modern AI workloads. By mitigating the energy burden associated with ADCs and enhancing dynamic range without energy penalties, the approach could facilitate more efficient hardware implementations for edge AI applications.

Key Findings