首页 > AI前沿 > Optimal approximation of smooth functions by ReLU neural networks

Optimal approximation of smooth functions by ReLU neural networks

arXiv机器学习 2026-10-16 06:00 3 阅读 查看原文

We study uniform approximation of the Hölder ball $\mathcal H_1^s([0,1]^D)$ by fully connected ReLU networks.

Except in the one-dimensional Lipschitz case, existing constructions remain separated from the information-theoretic efficiency limit.

This article closes the gap for every $s>0$ and $D\ge1$: networks with width $O(N)$, depth $O(L)$, and parameter magnitude $O(1)$ attain uniform approximation error of order $(N^2L^2\log N)^{-s/D}$. No relation between width and depth is required.

The construction uses new encoding and decoding mechanisms that exploit higher-order finite-difference bounds for local Taylor coefficients of the target function.

As a statistical consequence, empirical risk minimizers over the corresponding clipped network classes attain the minimax regression rate $n^{-2s/(2s+D)}$ without additional logarithmic factors.

The paper also gives a detailed exposition of the key approximation techniques, covering both established and new mechanisms.