De-identifying clinical notes with a hosted language model discloses the information the step is meant to protect. We present EdgeDeID, a 0.6B-parameter decoder-only de-identifier that runs locally and is trained without real patient text: three LLM teachers write synthetic notes, a single labeler re-annotates them into one convention, and the model learns to emit a typed XML schema of the identifiers it finds.
Performance
On a 100-note synthetic in-domain benchmark with manually verified annotations it reaches 0.994 micro-F1 and leaves no missed identifier in 99% of notes, against 0.857 and 12% for the strongest of eight off-the-shelf systems. Quantized, it runs on CPU in 615 MB.
Augmented training methods
- Only a verifier adapter helps: it separates complete from incomplete outputs of other systems at 0.94 AUROC.
- Reinforcement learning with a recall-weighted reward leaves in-domain results unchanged.
- A rule-based post-processing layer leaves in-domain results unchanged.
Domain shift to MIMIC ICU nursing notes
On MIMIC ICU nursing notes F1 falls to 0.48.
Error analysis and mitigation
The errors follow two properties our synthetic notes lack:
- Telegraphic style.
- Low identifier density.
Three hand-written in-context examples recover 0.08 F1 without annotated target data.