Many organizations adapt large pretrained models to their own tasks by fine-tuning on private data.
Several of these parties often hold data for the same task and wish to fine-tune a model together without pooling that data.
Federated learning (FL) enables joint fine-tuning, but reconstruction attacks on shared intermediate values (the model or its gradients) remain a privacy risk.
A one-shot protocol that exchanges one encrypted contribution exposes no intermediate value.
We present HE-OFT, the first cryptographically secure one-shot federated fine-tuning protocol in which no party receives the trained model.
Each client fine-tunes a low-rank adapter and a classifier head on a frozen public backbone and keeps the adapter.
The client uploads one encrypted head displacement, which the server combines under multiparty CKKS and never decrypts.
A quorum of clients returns only the predicted label to the querier.
Performance Results
On four text classification tasks and one vision task, HE-OFT reaches 61 to 79 per cent accuracy, against 20 to 48 per cent for a client training alone.
HE-OFT keeps 85 to 96 per cent of the accuracy of a disclosed model.
Query Time
A test-time query takes 443.1 to 1713.1 s on one core, or 56.1 to 255.1 s with level restoration on a GPU.
Restoring levels at the server cuts the traffic per query from up to 1.6 GiB to 13.5 MiB.