Prior-Data Fitted Networks (PFNs) replace the hyperparameter refitting of Gaussian Process (GP) surrogates in Bayesian optimisation with a single forward pass, at the cost of a prior that is fixed at meta-training time.
We show empirically that a PFN's predictive log-likelihood falls increasingly behind that of a refitted GP as the objective function's lengthscale grows beyond the mode of the PFN's meta-training prior, and that this deficit carries over to optimisation performance.
To correct the mismatch without retraining the network, we introduce Anisotropic Trust Region (ATR) warping.
- Per-dimension lengthscales are estimated by kernel target alignment.
- An affine rescaling of the input space maps the objective onto the lengthscale the PFN expects.
- A trust region around the anchor determines which observations are retained.
- This leaves the PFN's $\mathcal{O}(d \cdot N^2)$ inference cost unchanged.
On functions drawn from a Matérn-3/2 kernel, ATR warping improves optimisation performance of the PFN at both extremes of the lengthscale range.
Our implementation is publicly available at https://github.com/youngers2006/Trust-Region-Warping-For-PFNs.