首页 > AI前沿 > Training Leaves Traces: Centered Residual Signatures for Language Model Lineage Verification

Training Leaves Traces: Centered Residual Signatures for Language Model Lineage Verification

arXiv机器学习 2026-08-15 06:36 4 阅读 查看原文

Open-weight language models are fine-tuned, quantized, pruned, and merged, yet their provenance is often undocumented.

We study data-free white-box lineage verification: can weights alone reveal whether two compatible model checkpoints share ancestry?

Residual training produces a shared identity-aligned component in branch products, so this structure alone cannot establish ancestry.

We remove it and compare checkpoint-specific structure across residual blocks, yielding a symmetric lineage score calibrated against independent checkpoints.

On Benchmarks

On residual-MLP and GPT-2 benchmarks, the score separates fine-tuned, LoRA-merged, pruned, and quantized descendants from independent and distilled models (AUROC=1.0), distinguishing weight ancestry from behavioral similarity.

Experiments

Under function-preserving checkpoint laundering experiments, weight-space baselines lose margin or fail; our score remains unchanged and runs 76x faster than the nearest robust baseline on GPT-2.

The projection-pairing signal appears across six language-model families and beyond, and a case study correctly identifies 3 related and 7 unrelated LLaMA-2 public checkpoints.

Results

Collectively, these results establish a passive, data-free provenance signal for compatible open-weight language-model checkpoints.