Overview
This work presents a homogeneous tensor framework designed for high-order trust-region and spherical polynomial optimization. The framework addresses the challenges associated with solving nonconvex polynomial optimization problems, which arise in high-order methods for orders $p \geq 3$. The core approach involves a radius-controlled boundary mechanism based on homogeneous tensor representations. This representation allows any $p$-th order Taylor polynomial to be exactly modeled as an order-$p$ homogeneous tensor form by augmenting the step with a constant coordinate. At a predefined radius, this boundary model formulation results in a spherical polynomial optimization problem.
Research Context
High-order methods possess the potential to enhance worst-case evaluation complexity. However, for orders $p \geq 3$, these methods typically lead to Taylor subproblems that are nonconvex polynomial optimization problems. Such problems are generally difficult to solve. The development in this research aims to provide a structured way to handle these intrinsically difficult subproblems, particularly by leveraging homogeneous tensor representations.
Approach
The framework utilizes homogeneous tensor representations, where any $p$-th order Taylor polynomial can be precisely represented as an order-$p$ homogeneous tensor form via the augmentation of the step with a constant coordinate. This representation is applicable to arbitrary orders $p$. The algorithmic development within this research primarily concentrates on the cubic case, i.e., $p=3$.
For an inhomogeneous cubic on the sphere, a quadratic shift is introduced. Under an explicitly defined shift bound, the research proves an equivalence with a three-block multilinear formulation at global optimality. This equivalence motivated the development of a proximal alternating minimization (PAM) method.
- The PAM method features closed-form block updates.
- Its objective values are observed to decrease.
- Every accumulation point of the PAM method is stationary.
The boundary-step mechanism developed is embedded within an Adaptive Homogeneous Tensor Method (Ada–HTM). The Ada–HTM achieves an adaptive-regularization-type (AR$p$-type) evaluation complexity of $\mathcal{O}(\epsilon^{-(p+1)/p})$ for first-order stationarity. This complexity is attained under certain conditions, including explicit smoothness, safeguarded-decrease, weak-curvature nondegeneracy, and local-refinement conditions.
Findings
- The homogeneous tensor representation allows for the exact modeling of any $p$-th order Taylor polynomial as an order-$p$ homogeneous tensor form.
- For an inhomogeneous cubic on the sphere, a quadratic shift leads to equivalence with a three-block multilinear formulation at global optimality, given an explicit shift bound.
- The PAM method, derived from this formulation, exhibits decreasing objective values and stationary accumulation points due to its closed-form block updates.
- The Ada–HTM, which integrates the boundary-step mechanism, attains an evaluation complexity of $\mathcal{O}(\epsilon^{-(p+1)/p})$ for first-order stationarity under specified conditions.
- Numerically, the PAM method matches order-$2$ moment–sum-of-squares (SOS) certificates on structured cubic instances where certification is feasible.
- The PAM method demonstrates particular scaling efficiency for low-rank tensors.
- The Ada–HTM is competitive with established methods, such as trust-region and cubic-regularization methods.
- The largest gains for Ada–HTM are observed on ill-conditioned and badly-scaled problems.
Why This Matters
The developed framework offers a systematic approach to tackle the nonconvex polynomial optimization problems inherent in high-order optimization methods, particularly for orders $p \geq 3$. By providing a mechanism to solve these subproblems efficiently, the research contributes to the practical application of high-order methods to improve evaluation complexity. Its observed performance benefits for ill-conditioned and badly-scaled problems suggest its potential utility in challenging optimization scenarios.