Certifying Some Distributional Robustness with Principled Adversarial Training 1: SGD on the Wasserstein Penalty Surrogate Finds an Approximate Stationary Point at Rate 4σ√(L_φΔ_F/T)Research Paper
Neural networks trained by empirical risk minimization are sensitive to small, deliberately chosen perturbations of their inputs ("adversarial examples"). A standard defence is adversarial training: at each step, perturb the training example to increase the loss, then take a gradient step at the perturbed point. Sinha, Namkoong, Volpi and Duchi (arXiv:1710.1…