The Viscosity Subdifferential of the Rank Function via the Corresponding Subdifferential of its Moreau Envelopes
Jean-Baptiste Hiriart-Urruty
,
Hai Yen Le
Abstract
We derive the so-called viscosity subdifferential of the rank function via a limiting process applied to the Moreau envelopes of the rank function. Before that, we obtain the explicit expressions of all the generalized subdifferentials of the Moreau envelopes of the rank function.