Delayed Weighted Gradient Method with simultaneous step-sizes for strongly convex optimization

Hugo Lara; Rafael Aleixo; Harry Oviedo

doi:10.1007/s10589-024-00586-4

Delayed Weighted Gradient Method with simultaneous step-sizes for strongly convex optimization

Hugo Lara, Rafael Aleixo, Harry Oviedo

Research output: Contribution to journal › Article › peer-review

Abstract

The Delayed Weighted Gradient Method (DWGM) is a two-step gradient algorithm that is efficient for the minimization of large scale strictly convex quadratic functions. It has orthogonality properties that make it to compete with the Conjugate Gradient (CG) method. Both methods calculate in sequence two step-sizes, CG minimizes the objective function and DWGM the gradient norm, alongside two search directions defined over first order current and previous iteration information. The objective of this work is to accelerate the recently developed extension of DWGM to nonquadratic strongly convex minimization problems. Our idea is to define the step-sizes of DWGM in a unique two dimensional convex quadratic optimization problem, calculating them simultaneously. Convergence of the resulting algorithm is analyzed. Comparative numerical experiments illustrate the effectiveness of our approach.

Original language	English
Pages (from-to)	151-182
Number of pages	32
Journal	Computational Optimization and Applications
Volume	89
Issue number	1
DOIs	https://doi.org/10.1007/s10589-024-00586-4
State	Published - Sep 2024
Externally published	Yes

Keywords

Conjugate gradient methods
Gradient methods
Large-scale optimization
Strongly convex functions

Access to Document

10.1007/s10589-024-00586-4

Cite this

@article{e6b47d60db9d42c8a29fec223f1eaff8,

title = "Delayed Weighted Gradient Method with simultaneous step-sizes for strongly convex optimization",

abstract = "The Delayed Weighted Gradient Method (DWGM) is a two-step gradient algorithm that is efficient for the minimization of large scale strictly convex quadratic functions. It has orthogonality properties that make it to compete with the Conjugate Gradient (CG) method. Both methods calculate in sequence two step-sizes, CG minimizes the objective function and DWGM the gradient norm, alongside two search directions defined over first order current and previous iteration information. The objective of this work is to accelerate the recently developed extension of DWGM to nonquadratic strongly convex minimization problems. Our idea is to define the step-sizes of DWGM in a unique two dimensional convex quadratic optimization problem, calculating them simultaneously. Convergence of the resulting algorithm is analyzed. Comparative numerical experiments illustrate the effectiveness of our approach.",

keywords = "Conjugate gradient methods, Gradient methods, Large-scale optimization, Strongly convex functions",

author = "Hugo Lara and Rafael Aleixo and Harry Oviedo",

note = "Publisher Copyright: {\textcopyright} The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2024.",

year = "2024",

month = sep,

doi = "10.1007/s10589-024-00586-4",

language = "English",

volume = "89",

pages = "151--182",

journal = "Computational Optimization and Applications",

issn = "0926-6003",

publisher = "Springer Netherlands",

number = "1",

}

TY - JOUR

T1 - Delayed Weighted Gradient Method with simultaneous step-sizes for strongly convex optimization

AU - Lara, Hugo

AU - Aleixo, Rafael

AU - Oviedo, Harry

N1 - Publisher Copyright: © The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2024.

PY - 2024/9

Y1 - 2024/9

N2 - The Delayed Weighted Gradient Method (DWGM) is a two-step gradient algorithm that is efficient for the minimization of large scale strictly convex quadratic functions. It has orthogonality properties that make it to compete with the Conjugate Gradient (CG) method. Both methods calculate in sequence two step-sizes, CG minimizes the objective function and DWGM the gradient norm, alongside two search directions defined over first order current and previous iteration information. The objective of this work is to accelerate the recently developed extension of DWGM to nonquadratic strongly convex minimization problems. Our idea is to define the step-sizes of DWGM in a unique two dimensional convex quadratic optimization problem, calculating them simultaneously. Convergence of the resulting algorithm is analyzed. Comparative numerical experiments illustrate the effectiveness of our approach.

AB - The Delayed Weighted Gradient Method (DWGM) is a two-step gradient algorithm that is efficient for the minimization of large scale strictly convex quadratic functions. It has orthogonality properties that make it to compete with the Conjugate Gradient (CG) method. Both methods calculate in sequence two step-sizes, CG minimizes the objective function and DWGM the gradient norm, alongside two search directions defined over first order current and previous iteration information. The objective of this work is to accelerate the recently developed extension of DWGM to nonquadratic strongly convex minimization problems. Our idea is to define the step-sizes of DWGM in a unique two dimensional convex quadratic optimization problem, calculating them simultaneously. Convergence of the resulting algorithm is analyzed. Comparative numerical experiments illustrate the effectiveness of our approach.

KW - Conjugate gradient methods

KW - Gradient methods

KW - Large-scale optimization

KW - Strongly convex functions

UR - http://www.scopus.com/inward/record.url?scp=85194769292&partnerID=8YFLogxK

U2 - 10.1007/s10589-024-00586-4

DO - 10.1007/s10589-024-00586-4

M3 - Article

AN - SCOPUS:85194769292

SN - 0926-6003

VL - 89

SP - 151

EP - 182

JO - Computational Optimization and Applications

JF - Computational Optimization and Applications

IS - 1

ER -

Delayed Weighted Gradient Method with simultaneous step-sizes for strongly convex optimization

Abstract

Keywords

Access to Document

Fingerprint

Cite this