Large language models make weight storage and memory traffic major inference costs, motivating low-precision formats that represent each weight with only a few bits.
Schur Replay, A SCALE-SELECTION ALGORITHM THAT REPRODUCES THE GPTQ UPDATES CAUSED BY EACH BLOCK SCALE AND SCORES THE RESULTING BLOCK ERROR AFTER ACCOUNTING FOR COMPENSATION FROM UNQUANTIZED COLUMNS.