Variational Inference and Density Estimation with Non-Negative Hierarchical Tucker Format

Overview

This work introduces an efficient method for compressing high-dimensional discrete probability functions, also referred to as probability tensors, into a non-negative hierarchical Tucker (HT) format. The proposed methodology operates through a two-stage procedure designed for computational efficiency.

Approach

The methodology for compressing high-dimensional probability tensors involves two distinct stages:

1. In the first stage, an existing interpolation method is utilized to compress the target tensor. This compression transforms the tensor into a hierarchical Tucker format. The process is described as being similar to the CUR decomposition method, which is applied for low-rank matrix reconstruction.
2. The second stage involves fitting the output generated from the first stage. This fitting is performed against a non-negative hierarchical Tucker ansatz. A second-order method is employed for this fitting, specifically tailored for the non-negative hierarchical Tucker setting.

Both stages of this methodology exhibit a computational complexity of $\mathcal{O}(d)$ when the tensor is of order $d$. This computational efficiency is noted as facilitating the extension of the proposed methodology to high-dimensional settings.

Findings

• The developed two-stage procedure enables the compression of high-dimensional discrete probability functions into a non-negative hierarchical Tucker format.
• The computational complexity for both stages of the methodology is $\mathcal{O}(d)$ where $d$ is the order of the tensor. This characteristic allows the methodology to scale effectively to high-dimensional problems.
• Numerical experiments conducted using the method demonstrated success in compressing various high-dimensional probability tensors.

Why This Matters

The ability to efficiently compress high-dimensional discrete probability functions is relevant for managing and analyzing complex probabilistic models. The $\mathcal{O}(d)$ computational complexity suggests the method's applicability to scenarios involving many variables.