研究概述
In this article, we study deep nonparametric regression trained via stochastic gradient Langevin dynamics (SGLD), a variant of stochastic gradient descent that injects carefully calibrated Gaussian noise during optimization.
理论结果（PAC-Bayesian，Bernstein 条件）
Using a PAC-Bayesian analysis tailored to deep ReLU networks, we establish an excess-risk bound that simultaneously accounts for network approximation error, statistical complexity, and the algorithmic cost accumulated along the SGLD trajectory. Under a Bernstein condition and within a stable SGLD regime, the excess-risk bound contains a stochastic term whose coefficient is inversely proportional to the variance of injected noise, while the estimator retains the optimal rate for nonparametric regression under the H\"older smoothness condition up to logarithmic factors.
对“适度噪声”现象的视角
This result provides an approximation-error-aware, rate-optimal statistical perspective on the widely observed generalization benefits of moderate SGLD noise.
逐样本裁剪 SGLD 的隐私保证
For per-sample clipped SGLD, we further establish an exact f-DP guarantee and the corresponding asymptotic $\mu$-GDP guarantee for the proposed estimator.
DP 保证的选择
We establish statistical convergence rates and differential privacy guarantees for DNN estimators trained using SGLD, while explicitly accounting for both network approximation error and the effect of injected noise in the excess-risk bound.
方法学贡献
- 将网络近似误差、统计复杂度与沿 SGLD 轨迹累积的算法代价同时纳入超额风险界。
- 在 Bernstein 条件下给出对注入噪声方差成反比的随机项，并在 H\"older 光滑假设下达到（至多仅损失对数因子的）非参数回归最优速率。
- 为逐样本裁剪 SGLD 建立精确 f-DP 保证，并给出相应的渐近 $\mu$-GDP 保证。
实验评估
Extensive simulation studies and analyses of two real datasets examine the finite-sample predictive performance and privacy-utility behavior of the proposed procedure.