Learning with smooth Hinge losses

[摘要] Due to the non-smoothness of the Hinge loss in SVM, it is difficult to obtain a faster convergence rate with modern optimization algorithms. In this paper, we introduce two smooth Hinge losses psi(G)(alpha, sigma) and psi(M)(alpha; sigma) which are infinitely differentiable and converge to the Hinge loss uniformly in alpha as sigma tends to 0. By replacing the Hinge loss with these two smooth Hinge losses, we obtain two smooth support vector machines (SSVMs), respectively. Solving the SSVMs with the Trust Region Newton method (TRON) leads to two quadratically convergent algorithms. Experiments in text classification tasks show that the proposed SSVMs are effective in real-world applications. We also introduce a general smooth convex loss function to unify several commonly-used convex loss functions in machine learning. The general framework provides smooth approximation functions to non-smooth convex loss functions, which can be used to obtain smooth models that can be solved with faster convergent optimization algorithms. (C) 2021 Elsevier B.V. All rights reserved.

[发布日期] 2021-11-06

[关键词] Smooth Hinge loss;Convex surrogate loss;Support vector machine;Trust region Newton method