,有需要可以加h2/h3?不对,先看原文的结构,原文是一整段?不对,先拆分自然段?不对,原文的内容:首先第一部分是介绍QLoRA主导,然后提出问题,然后讲方法,然后理论贡献,然后实验结果,然后部署相关的。不对,要保留所有信息,不能增删改。首先,先看原文是几个自然段?哦,原文给的正文是一整段?不对,仔细看:原文的正文是连续的,但我们可以合理划分?不对,不对,规则说每个自然段落用
包裹,那原文的原始自然段落,哦,用户给的正文是一段?不对,等下看用户给的正文:从开头到结尾是一整段?不对,我们再读: 首先,第一部分:Parameter-efficient fine-tuning of quantized large language models is dominated by QLoRA, which combines 4-bit weight quantization with FP16 low-rank adapters. We ask whether the FP16 adapter precision is necessary. Replacing it with a single binary bit per adapter parameter, combined with a learned per-layer scale $s$ and a Scaled Straight-Through Estimator, reduces adapter storage by 13-16x. We introduce two variants -- NormScale, which binarizes under a norm-preserving scale with a clipped STE, and LearnThresh, which makes the binarization threshold learnable -- unified under a framework we call BQ-LoRA. 这部分是介绍背景和提出的方法框架,可以作为一个
?然后下一部分是理论贡献:Our main theoretical contribution is a tight characterization of the BQ-LoRA hypothesis class: it has bounded entrywise norm, a typical-set spectral norm bound that is $Θ(r)$ times sharper than the worst case, a full-rank, near-isotropic update, and consequently a Rademacher complexity that yields an explicit $\tilde{O}(s_{\max}α\sqrt{d_{out}/(rn)})$ generalization bound. FP16 LoRA has no comparable a priori norm or isotropy control. 这部分可以作为第二个
?然后下一部分是实验结果:On Qwen2.5 (1.5B-72B), BQ-LoRA exceeds QLoRA's benchmark averages at every scale by 0.8-3.5 points; on LLaMA-2 (7B/70B) it remains competitive at the same storage reduction but does not uniformly lead the strongest FP16-adapter baselines (LoftQ at 7B, QLoRA at 70B). On Qwen2.5-7B the binary adapter preserves the 4-bit base WikiText-2 PPL within 0.06 (9.86 vs. 9.80) while QLoRA worsens it by 0.59. 这部分作为第三个
?然后下一部分是资源和部署:The reduction applies to inference-time adapter storage and serving; training memory matches QLoRA and step time rises by under 4%. We translate the storage saving into a deployment study, BQ-SGMV, by extending Punica's SGMV kernel with a binary inner-product primitive: the same A100-80GB holds 15.6x more adapters in HBM, and our research prototype eliminates cache misses under a skewed synthetic workload. 这部分作为第四个
?不对,要不要加h2或者h3?比如有没有小节?哦,原文的内容逻辑是:背景与方法提出、理论贡献、实验结果、部署应用,这四个小节?那可以用h3来划分?不对,规则说长文用h2/h3划分小节,那我们看看: 首先,第一个小节是
背景与方法框架
?不对不对,规则说不得增删改内容!哦对!规则4是保留原文所有信息,不得增删改内容,那不能加原文没有的文字啊!哦,对哦,我刚才差点加了小标题的文字,不行,因为原文没有这些,不能加。那哦,原来如此,不能加原文没有的内容,那h2/h3是划分小节,但小节的标题必须是原文有的?不对,原文没有小节标题啊,那原文就是一段?不对,不对,再仔细看用户给的正文:用户给的正文是一整段吗?看: 用户给的正文: Parameter-efficient fine-tuning of quantized large language models is dominated by QLoRA, which combines 4-bit weight quantization with FP16 low-rank adapters. We ask whether the FP16 adapter precision is necessary. Replacing it with a single binary bit per adapter parameter, combined with a learned per-layer scale $s$ and a Scaled Straight-Through Estimator, reduces adapter storage by 13-16x. We introduce two variants -- NormScale, which binarizes under a norm-preserving scale with a clipped STE, and LearnThresh, which makes the binarization threshold learnable -- unified under a framework we call BQ-LoRA. Our main theoretical contribution is a tight characterization of the BQ-LoRA hypothesis class: it has bounded entrywise norm, a typical-set spectral norm bound that is $Θ(r)$ times sharper than the worst case, a full-rank, near-isotropic update, and consequently a Rademacher complexity that yields an explicit $\tilde{O}(s_{\max}α\sqrt{d_{out}/(rn)})$ generalization bound. FP16 LoRA has no comparable a priori norm or isotropy control. On Qwen2.5 (1.5B-72B), BQ-LoRA exceeds QLoRA's benchmark averages at every scale by 0.8-3.5 points; on LLaMA-2 (7B/70B) it remains competitive at the same storage reduction but does not uniformly lead the strongest FP16-adapter baselines (LoftQ at 7B, QLoRA at 70B). On Qwen2.5-7B the binary adapter preserves the 4-bit base WikiText-2 PPL within 0.06 (9.86 vs. 9.80) while QLoRA worsens it by 0.59. The reduction applies to inference-time adapter storage and serving; training memory matches QLoRA and step time rises by under 4%. We translate the storage saving into a deployment study, BQ-SGMV, by extending Punica's SGMV kernel with a binary inner-product primitive: the same A100-80GB holds 15.6x more adapters in HBM, and our research prototype eliminates cache misses under a skewed synthetic workload. 哦,原来这是一整段?不对,但我们可以根据语义拆分自然段落吗?规则说“每个自然段落用包裹”,这里的自然段落是指原文的段落,还是语义上的段落?不对,规则4是不得增删改内容,那拆分段落是排版,不算改内容对吧?因为内容还是那些,只是分段。那现在,我们可以把内容按语义分成几个
,重要的地方用?比如关键的术语?比如QLoRA,BQ-LoRA这些可以用?不对,规则说重要句子可用,也可以重要的术语?看规则3:重要句子可用。那我们来梳理: 首先,第一个
:讲背景和问题,然后提出方法的核心:
Parameter-efficient fine-tuning of quantized large language models is dominated by QLoRA, which combines 4-bit weight quantization with FP16 low-rank adapters. We ask whether the FP16 adapter precision is necessary. Replacing it with a single binary bit per adapter parameter, combined with a learned per-layer scale $s$ and a Scaled Straight-Through Estimator, reduces adapter storage by 13-16x. We introduce two variants -- NormScale, which binarizes under a norm-preserving scale with a clipped STE, and LearnThresh, which makes the binarization threshold learnable -- unified under a framework we call BQ-LoRA.
然后第二个讲理论贡献:
Our main theoretical contribution is a tight characterization of the BQ-LoRA hypothesis class: it has bounded entrywise norm, a typical-set spectral norm bound that is $Θ(r)$ times sharper than the worst case, a full-rank, near-isotropic update, and consequently a Rademacher complexity that yields an explicit $\tilde{O}(s_{\max