Stationarity and Convergence in Reduce-or-Retreat Minimization

The book presents a comprehensive framework for various numerical optimization methods through its "reduce-or-retreat" approach, which balances reducing the objective at trial points with retreating to closer alternatives. It aligns derivative-based methods, fostering the development of new techniques and theoretical insights. The author introduces generalizations for non-smooth objectives and examines convergence through a novel notion of stationarity. By exploring descent conditions and situational convergence, the text broadens traditional analyses and connects diverse minimization strategies.