A comparison of the convergence rates of Hestenes’ conjugate Gram-Schmidt method without derivatives with other numerical optimization methods

[1] Brent R.P., Algorithms for Minimization Without Derivatives, Courier Corporation, Dover Publication, New York, USA, 2013. BrentR.P. Algorithms for Minimization Without Derivatives Courier Corporation, Dover Publication New York, USA 2013 Search in Google Scholar

[2] Wu X., Shao H., Liu P., Zhang Y., Zhuo Y., An efficient conjugate gradient-based algorithm for unconstrained optimization and its projection extension to large-scale constrained nonlinear equations with applications in signal recovery and image denoising problems, Journal of Computational and Applied Mathematics, 422, 114879, 2023. WuX. ShaoH. LiuP. ZhangY. ZhuoY. An efficient conjugate gradient-based algorithm for unconstrained optimization and its projection extension to large-scale constrained nonlinear equations with applications in signal recovery and image denoising problems Journal of Computational and Applied Mathematics 422 114879 2023 Search in Google Scholar

[3] Davidon W.C., Variable metric method for minimization, Argonne National Laboratory, Lemont, Illinois, (Master Thesis), ANL-5990, University of Chicago, USA, 1959. DavidonW.C. Variable metric method for minimization, Argonne National Laboratory, Lemont, Illinois (Master Thesis), ANL-5990, University of Chicago USA 1959 Search in Google Scholar

[4] Fletcher R., Powell M.J.D., A rapidly convergent descent method for minimization, The Computer Journal, 6(2), 163–168, 1963. FletcherR. PowellM.J.D. A rapidly convergent descent method for minimization The Computer Journal 6 2 163 168 1963 Search in Google Scholar

[5] Hestenes M.R., Stiefel E., Methods of conjugate gradients for solving linear systems, Journal of Research of the National Bureau of Standards, 49(6), 409–436, 1952. HestenesM.R. StiefelE. Methods of conjugate gradients for solving linear systems Journal of Research of the National Bureau of Standards 49 6 409 436 1952 Search in Google Scholar

[6] Fletcher R., Reeves C.M., Function minimization by conjugate gradients, The Computer Journal, 7(2), 149–154, 1964. FletcherR. ReevesC.M. Function minimization by conjugate gradients The Computer Journal 7 2 149 154 1964 Search in Google Scholar

[7] Nocedal J., Wright S.J., Numerical Optimization (2nd Ed.), Springer, USA, 2006. NocedalJ. WrightS.J. Numerical Optimization 2nd Ed. Springer USA 2006 Search in Google Scholar

[8] Yousif O.O.O., Mohammed M.A.Y., Saleh M.A., Elbashir M.K., A criterion for the global convergence of conjugate gradient methods under strong Wolfe line search, Journal of King Saud University-Science, 34(8), 102281, 2022. YousifO.O.O. MohammedM.A.Y. SalehM.A. ElbashirM.K. A criterion for the global convergence of conjugate gradient methods under strong Wolfe line search Journal of King Saud University-Science 34 8 102281 2022 Search in Google Scholar

[9] Kelley C.T., Iterative Methods for Optimization, SIAM, Philadelphia, USA, 1999. KelleyC.T. Iterative Methods for Optimization SIAM Philadelphia, USA 1999 Search in Google Scholar

[10] Zhang L., An improved Wei-Yao-Liu nonlinear conjugate gradient method for optimization computation, Applied Mathematics and Computation, 215(6), 2269–2274, 2009. ZhangL. An improved Wei-Yao-Liu nonlinear conjugate gradient method for optimization computation Applied Mathematics and Computation 215 6 2269 2274 2009 Search in Google Scholar

[11] Doikov N., Nesterov Y., Gradient regularization of Newton method with Bregman distances, Mathematical Programming, DOI:10.1007/s10107-023-01943-7, 2023. DoikovN. NesterovY. Gradient regularization of Newton method with Bregman distances Mathematical Programming 10.1007/s10107-023-01943-7 2023 Open DOISearch in Google Scholar

[12] Stanimirović P.S., Shaini B.I., Sabi’u J., Shah A., Petrović M.J., Ivanov B., Cao X., Stupina A., Li S., Improved gradient descent iterations for solving systems of nonlinear equations, Algorithms, 16(64), 1–23, 2023. StanimirovićP.S. ShainiB.I. Sabi’uJ. ShahA. PetrovićM.J. IvanovB. CaoX. StupinaA. LiS. Improved gradient descent iterations for solving systems of nonlinear equations Algorithms 16 64 1 23 2023 Search in Google Scholar

[13] Gupta S.D., Freund R.M., Sun X.A., Taylor A., Nonlinear conjugate gradient methods: worst-case convergence rates via computer-assisted analyses, arXiv:2301.01530, 2023. GuptaS.D. FreundR.M. SunX.A. TaylorA. Nonlinear conjugate gradient methods: worst-case convergence rates via computer-assisted analyses arXiv:2301.01530, 2023 Search in Google Scholar

[14] Ortega J.M., Rheinboldt W.C., Iterative solution of nonlinear equations in several variables, SIAM, Philadelphia, USA, 1999. OrtegaJ.M. RheinboldtW.C. Iterative solution of nonlinear equations in several variables SIAM Philadelphia, USA 1999 Search in Google Scholar

[15] Hestenes M.R., Conjugate Direction Methods in Optimization (Vol:12), Springer, New York, USA, 1980. HestenesM.R. Conjugate Direction Methods in Optimization 12 Springer New York, USA 1980 Search in Google Scholar

[16] Russak I.B., Convergence of the conjugate Gram-Schmidt method, Journal of Optimization Theory and Applications, 33, 163–173, 1981. RussakI.B. Convergence of the conjugate Gram-Schmidt method Journal of Optimization Theory and Applications 33 163 173 1981 Search in Google Scholar

[17] Stein J.I., Conjugate Direction Algorithms in Numerical Analysis and Optimization, (Final Report), U.S. Army Research Office, DAHC 04-74-G-0006, National Science Foundation GP-40175, The University of Toledo, Toledo, Ohio, USA, 1975. SteinJ.I. Conjugate Direction Algorithms in Numerical Analysis and Optimization (Final Report), U.S. Army Research Office, DAHC 04-74-G-0006, National Science Foundation GP-40175 The University of Toledo, Toledo, Ohio, USA 1975 Search in Google Scholar

[18] Stein J.I., Raihen M.N., Convergence rates for Hestenes’ Gram-Schmidt conjugate direction method without derivatives in numerical optimization, AppliedMath, 3(2), 268–285, 2023. SteinJ.I. RaihenM.N. Convergence rates for Hestenes’ Gram-Schmidt conjugate direction method without derivatives in numerical optimization AppliedMath 3 2 268 285 2023 Search in Google Scholar

[19] Dennemeyer R.F., Mookini E.H., CGS algorithms for unconstrained minimization of functions, Journal of Optimization Theory and Applications, 16(1–2), 67–85, 1975. DennemeyerR.F. MookiniE.H. CGS algorithms for unconstrained minimization of functions Journal of Optimization Theory and Applications 16 1–2 67 85 1975 Search in Google Scholar

[20] Ortega J.M., Rheinboldt W.C., On a class of approximate iterative processes, Archive for Rational Mechanics and Analysis, 23, 352–365, 1967. OrtegaJ.M. RheinboldtW.C. On a class of approximate iterative processes Archive for Rational Mechanics and Analysis 23 352 365 1967 Search in Google Scholar

[21] Raihen N., Convergence Rates for Hestenes’ Gram-Schmidt Conjugate Direction Method without Derivatives in Numerical Optimization, (Masters Thesis), University of Toledo, Toledo, USA, 2017. RaihenN. Convergence Rates for Hestenes’ Gram-Schmidt Conjugate Direction Method without Derivatives in Numerical Optimization (Masters Thesis), University of Toledo Toledo, USA 2017 Search in Google Scholar

[22] Andrei N., Nonlinear Conjugate Gradient Methods for Unconstrained Optimization, Springer, Switzerland, 2020. AndreiN. Nonlinear Conjugate Gradient Methods for Unconstrained Optimization Springer Switzerland 2020 Search in Google Scholar

[23] Olsen N.C., Private Communication to Ivie Stein Jr., Consultant, Lockheed, Palmdale, California, USA, 2005. OlsenN.C. Private Communication to Ivie Stein Jr., Consultant Lockheed Palmdale, California, USA 2005 Search in Google Scholar

[24] Desrochers A., Mohseni S., Quadratic optimization via conjugate directions and projection matrices, 1985 American Control Conference, IEEE, 19–21 June 1985, Boston, Massachusetts, USA, 1684–1688, 1985. DesrochersA. MohseniS. Quadratic optimization via conjugate directions and projection matrices 1985 American Control Conference, IEEE 19–21 June 1985 Boston, Massachusetts, USA 1684 1688 1985 Search in Google Scholar

[25] Stein I., Conjugate gradient methods in Banach spaces, Nonlinear Analysis: Theory Methods and Applications, 63(5–7), e2621–e2628, 2005. SteinI. Conjugate gradient methods in Banach spaces Nonlinear Analysis: Theory Methods and Applications 63 5–7 e2621 e2628 2005 Search in Google Scholar