Most recent post:
04 Sep 2020
BFGS stands for Broyden–Fletcher–Goldfarb–Shanno algorithm [1] and it’s a non-linear numerical optimization method. L-BFGS means Low Memory BFGS and is a variant of the original algorithm that uses less memory [2]. The problem it’s aiming to solve is to mimize a given function \(f: R^d \rightarrow R\) (this is applicable to a maximization problem - we just need to solve for \(-f\)).
In this post we’ll cover 5 topics: Taylor Expansion, Newton’s method, QuasiNewton methods, BFGS and L-BFGS. We then look back to see how all these fit together in the big picture.