Overview
This study examines unconstrained optimization methods that employ local quadratic models, regularized by a power $r \ge 3$ of the norm of the step. The focus is on scenarios where the objective function and its gradient are the sole evaluated components. The research establishes that, with a precise selection of the regularized model during each iteration, the complete sequence of generated points converges when the objective function exhibits pseudoconvexity. This outcome is attained by utilizing appropriate matrices to ensure the sequence of points maintains a variable metric quasi-Fejér monotone property.
Research Context
In optimization algorithms, the sequence of points generated frequently exhibits multiple limit points. However, within the context of convexity assumptions, (sub)gradient methods are recognized for producing a convergent sequence of points. This property, typically associated with convexity, is extended in the present work to encompass a broader category of algorithms.
Approach
The research investigates unconstrained optimization methods that utilize local quadratic models. A key characteristic of these models is their regularization by a power $r \ge 3$ of the norm of the step. The methodology centers on situations where the information available to the algorithm is limited to the objective function and its gradient. The convergence of the sequence of points is shown to be contingent on a careful selection of the regularized model at each iteration. The analytical framework relies on the employment of specific matrices designed to ensure that the generated sequence of points satisfies the condition of being variable metric quasi-Fejér monotone.
Findings
The analysis indicates that for the class of unconstrained optimization methods studied, the entire sequence of generated points converges. This convergence is observed under the condition that the objective function is pseudoconvex. The mechanism underpinning this convergence is the judicious choice of the regularized model in every iteration, combined with the use of appropriate matrices. These matrices are instrumental in establishing a variable metric quasi-Fejér monotone property for the sequence of points.