Our paper "Fair-GPTQ: Bias-Aware Quantization for Large Language Models" is accepted to TACL journal!

We show that quantization can amplify biases in LLMs and introduce Fair-GPTQ, a bias-aware quantization method that reduces unfairness while preserving model performance and the efficiency benefits of 4-bit quantization. Link for the paper: https://arxiv.org/abs/2509.15206
Code: https://github.com/upunaprosk/fair-gptq

The high memory demands of generative language models have drawn attention to quantization, which reduces memory usage by mapping model weights to lower-precision integers. However, recent empirical studies show that, while efficient, quantization can increase the likelihood of generating biased outputs and degrade performance on fairness benchmarks.

To address this issue, we introduce Fair-GPTQ, a post-training quantization method that incorporates explicit group-fairness constraints into the quantization objective. By extending the reconstruction-based optimization of GPTQ with a bias-aware regularization term, our method guides the rounding operation toward less-biased text generation for protected groups.

We focus on stereotype generation involving occupational bias and discriminatory language spanning gender, race, and religion. Our experiments demonstrate that Fair-GPTQ reduces unfairness relative to a half-precision model while preserving at least 90% of baseline accuracy on zero-shot benchmarks and retaining the memory and speed benefits of 4-bit quantization.

Contributions

(i) We introduce Fair-GPTQ, a bias-aware post-training quantization method that incorporates explicit group-fairness constraints into the quantization objective. We derive a theoretical solution to the constrained optimization problem with a group-bias term, extending the reconstruction-based approach of GPTQ.

(ii) We evaluate Fair-GPTQ on generative LLMs across fairness and performance benchmarks, showing that it reduces biased outputs while preserving model performance and the efficiency benefits of low-bit quantization.

(iii) We investigate the impact of quantization on model fairness at the channel and weight levels, demonstrating how our approach can be used to analyze the contributions of individual model components to fairness during quantization.

Our findings demonstrate that incorporating fairness constraints directly into the quantization objective can improve the fairness-performance trade-off, providing a framework for bias-aware compression of large language models.