Valentina Cammarota, Domenico Marinucci, Michele Salvi, Stefano Vigogna
tlooto Summary
A quantitative functional central limit theorem is proved for one-hidden-layer neural networks with generic activation function with rates of convergence which range from logarithmic for nondifferentiable nonlinearities such as the ReLu to $\sqrt{n}$ for highly regular activations.
Abstract
We prove a quantitative functional central limit theorem for one-hidden-layer neural networks with generic activation function. Our rates of convergence depend heavily on the smoothness of the activation function, and they range from logarithmic for nondifferentiable nonlinearities such as the ReLu to $\sqrt{n}$ for highly regular activations. Our main tools are based on functional versions of the Stein–Malliavin method; in particular, we rely on a quantitative functional central limit theorem which has been recently established by Bourguin and Campese [Electron. J. Probab. 25 (2020), 150].
Citation format
CAMMAROTA, Valentina, et al. A quantitative functional central limit theorem for shallow neural networks [preprint]. arXiv, 2023. arXiv:2306.16932.