CORS / Optimization Days
HEC Montréal, May 2931, 2023
CORSJOPT2023
HEC Montreal, 29 — 31 May 2023
NOJII Numerical optimization and linear algebra with Julia II
May 29, 2023 03:30 PM – 05:10 PM
Location: Transat (yellow)
Chaired by Tangi Migot
3 Presentations

03:30 PM  03:55 PM
Hyperplane Arrangements and Matroids in Complementarity Problems: the Bdifferential of the componentwise Minimum
Systems of smooth nonlinear equations of the form H(x) = 0 can often be
solved by the famous Newton method. When the equations involve nonsmooth
functions, this algorithm is no longer adapted. The semismooth Newton
method is a possible remedy to overcome this difficulty: it yields fast
local convergence, provided the Bdifferential of H has only nonsingular
Jacobians at the considered solution.An example of nonsmooth vector function H is encountered when
complementarity problems are reformulated as nonsmooth equations
thanks to the componentwise minimum function. This talk describes
the content and properties of the Bdifferential of the componentwise
minimum of two affine vector functions.Describing and computing this Bdifferential is a problem with
connections to linear algebra, convex anaysis and discrete geometry
(hyperplane arrangement), the latter being the approach chosen numerically.
We shall emphasize the role of duality and matroid circuits in these links,
and show how insight acquired by the theory serves the numerical aspect.The presentation will show significant improvements on the state of the art
approach by Rada and Černý that solves numerically this combinatorial
problem. 
03:55 PM  04:20 PM
PartiallySeparableNLPModels.jl: A Julia framework for partitioned quasiNewton optimization
In optimization applications, objectives and constraints often have a partiallyseparable structure: they are a sum of socalled element functions that depend on a small number of variables.
It is well known that twicedifferentiable functions with a sparse Hessian are partiallyseparable.
We present a framework based on the JuliaSmoothOptimizers ecosystem to model and detect partiallyseparable problems, and to efficiently evaluate their derivatives by informing AD engines of their structure.
We also implement partitioned trustregion quasiNewton methods in which Hessian approximations are also informed by the partiallyseparable structure.
Such methods were proposed in the 1980s and have rivaled limitedmemory quasiNewton methods in efficiency since.
Our implementations feature novel aspects, including the automatic detection of convexity and its impact on the choice of a quasiNewton approximation, and the use of limitedmemory approximations for element functions that depend on a sufficiently large number of variables, which consumes significantly less memory than the dense matrices used in the 1980s and 1990s.
Our numerical results illustrate the performance of a number of variants on a standard problem collection and demonstrate the viability of our methods for problems in which element functions depend on both a small or a substantial number of variables. 
04:20 PM  04:45 PM
Quadratic Regularization Optimizer in Low Precision for Deep Neural Networks: Implementation and Numerical Experience
We propose a Julia framework to automatically port a deterministic optimization method to the stochastic setting for training deep neural networks (DNNs). We illustrate the process using the quadratic regularization method R2, which is a variant of the gradient method with adaptive step size. We report on numerical experiments using the stochastic R2 method on classification networks in lowprecision arithmetic.