Awesome Model Quantization

This repo collects papers, documents, and codes about model quantization for anyone who wants to research it. We are continuously improving the project. Use it to navigate the topic and choose relevant methods, papers or tools.

Editorial note: curated source snapshot published by Collider.club under the MIT License. Source attribution is preserved in the front matter.

Source snapshot

Awesome Model Quantization

This repo collects papers, documents, and codes about model quantization for anyone who wants to research it. We are continuously improving the project. Welcome to PR the works (papers, repositories) that the repo misses.

• Benchmarks
• Survey Papers
• Papers
  • 2026
  • 2025
  • 2024
  • 2023
  • 2022–2015
• Books
• Related Repositories

Benchmarks

1. BiBench: Benchmarking and Analyzing Network Binarization [Paper] [Code]

Venue: ICML 2023

Authors: Haotong Qin, Mingyuan Zhang, Yifu Ding, Aoyu Li, Zhongang Cai, Ziwei Liu, Fisher Yu, Xianglong Liu.

@inproceedings{qin2023bibench,
  title={BiBench: Benchmarking and Analyzing Network Binarization},
  author={Qin, Haotong and Zhang, Mingyuan and Ding, Yifu and Li, Aoyu and Cai, Zhongang and Liu, Ziwei and Yu, Fisher and Liu, Xianglong},
  booktitle={International Conference on Machine Learning (ICML)},
  year={2023}
}

2. An empirical study of LLaMA3 quantization: from LLMs to MLLMs [Paper] [Code]

Venue: Visual Intelligence 2024

Authors: Wei Huang, Xingyu Zheng, Xudong Ma, Haotong Qin, Chengtao Lv, Hong Chen, Jie Luo, Xiaojuan Qi, Xianglong Liu, Michele Magno.

@article{huang2024empirical,
  title={An empirical study of llama3 quantization: From llms to mllms},
  author={Huang, Wei and Zheng, Xingyu and Ma, Xudong and Qin, Haotong and Lv, Chengtao and Chen, Hong and Luo, Jie and Qi, Xiaojuan and Liu, Xianglong and Magno, Michele},
  journal={Visual Intelligence},
  volume={2},
  number={1},
  pages={36},
  year={2024},
  publisher={Springer}
}

3. An Empirical Study of Qwen3 Quantization [Paper] [Code]

Venue: Visual Intelligence 2026

Authors: Xingyu Zheng, Yuye Li, Haoran Chu, Yue Feng, Xudong Ma, Jie Luo, Jinyang Guo, Haotong Qin, Michele Magno, Xianglong Liu.

@article{zheng2025empirical,
  title={An empirical study of qwen3 quantization},
  author={Zheng, Xingyu and Li, Yuye and Chu, Haoran and Feng, Yue and Ma, Xudong and Luo, Jie and Guo, Jinyang and Qin, Haotong and Magno, Michele and Liu, Xianglong},
  journal={arXiv preprint arXiv:2505.02214},
  year={2025}
}

4. LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit [Paper] [Code]

Venue: EMNLP 2024 Industry Track

Authors: Ruihao Gong, Yang Yong, Shiqiao Gu, Yushi Huang, Chengtao Lv, Yunchen Zhang, Xianglong Liu, Dacheng Tao.

@inproceedings{gong2024llmc,
  title={Llmc: Benchmarking large language model quantization with a versatile compression toolkit},
  author={Gong, Ruihao and Yong, Yang and Gu, Shiqiao and Huang, Yushi and Lv, Chengtao and Zhang, Yunchen and Tao, Dacheng and Liu, Xianglong},
  booktitle={Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track},
  pages={132--152},
  year={2024}
}

5. RobustMQ: Benchmarking Robustness of Quantized Models [Paper]

Venue: Visual Intelligence 2023

Authors: Yisong Xiao, Aishan Liu, Tianyuan Zhang, Haotong Qin, Jinyang Guo, Xianglong Liu.

@article{xiao2023robustmq,
  title={Robustmq: benchmarking robustness of quantized models},
  author={Xiao, Yisong and Liu, Aishan and Zhang, Tianyuan and Qin, Haotong and Guo, Jinyang and Liu, Xianglong},
  journal={Visual Intelligence},
  volume={1},
  number={1},
  pages={30},
  year={2023},
  publisher={Springer}
}

Survey Papers

1. Binary Neural Networks: A Survey [Paper] [Blog]

Venue: Pattern Recognition 2020

Authors: Haotong Qin, Ruihao Gong, Xianglong Liu, Xiao Bai, Jingkuan Song, Nicu Sebe.

@article{Qin:pr20_bnn_survey,
    title = "Binary neural networks: A survey",
    author = "Haotong Qin and Ruihao Gong and Xianglong Liu and Xiao Bai and Jingkuan Song and Nicu Sebe",
    journal = "Pattern Recognition",
    volume = "105",
    pages = "107281",
    year = "2020"
}

2. A Survey of Low-bit Large Language Models: Basics, Systems, and Algorithms [Paper]

Venue: Neural Networks 2025

Authors: Ruihao Gong, Yifu Ding, Zining Wang, Chengtao Lv, Xingyu Zheng, Jinyang Du, Yang Yong, Shiqiao Gu, Haotong Qin, Jinyang Guo, Dahua Lin, Michele Magno, Xianglong Liu.

@article{gong2025survey,
  title={A survey of low-bit large language models: Basics, systems, and algorithms},
  author={Gong, Ruihao and Ding, Yifu and Wang, Zining and Lv, Chengtao and Zheng, Xingyu and Du, Jinyang and Yong, Yang and Gu, Shiqiao and Qin, Haotong and Guo, Jinyang and others},
  journal={Neural networks},
  pages={107856},
  year={2025},
  publisher={Elsevier}
}

3. Low-bit Model Quantization for Deep Neural Networks: A Survey [Paper]

Venue: arXiv 2025

Authors: Kai Liu, Qian Zheng, Kaiwen Tao, Zhiteng Li, Haotong Qin, Wenbo Li, Yong Guo, Xianglong Liu, Linghe Kong, Guihai Chen, Yulun Zhang, Xiaokang Yang.

@article{liu2025low,
  title={Low-bit model quantization for deep neural networks: A survey},
  author={Liu, Kai and Zheng, Qian and Tao, Kaiwen and Li, Zhiteng and Qin, Haotong and Li, Wenbo and Guo, Yong and Liu, Xianglong and Kong, Linghe and Chen, Guihai and others},
  journal={arXiv preprint arXiv:2505.05530},
  year={2025}
}

Papers

2026

• [arXiv] EdgeRazor: A Lightweight Framework for Large Language Models via Mixed-Precision Quantization-Aware Distillation [code]
• [ICLR] PT²-LLM: Post-Training Ternarization for Large Language Models [code]
• [ICLR] Quant-dLLM: Post-Training Extreme Low-Bit Quantization for Diffusion Large Language Models
• [ICLR] DVD-Quant: Data-free Video Diffusion Transformers Quantization
• [ICLR] Q&C: When Quantization Meets Cache in Efficient Generation
• [CVPR Findings] Q-MambaIR: Accurate Quantized Mamba for Efficient Image Restoration
• [ICLR] Quantized Visual Geometry Grounded Transformer
• [ICLR] Post-Training Quantization for Video Matting
• [ICLR] QVGen: Pushing the Limit of Quantized Video Generative Models
• [ICLR] QuantSparse: Comprehensively Compressing Video Diffusion Transformer with Model Quantization and Attention Sparsification [code]
• [AAAI] First-Order Error Matters: Accurate Compensation for Quantized Large Language Models [code]
• [AAAI] TR-DQ: Time-Rotation Diffusion Quantization
• [ICLR] TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate
• [ICLR] Optimal Brain Restoration for Joint Quantization and Sparsification of LLMs [code]
• [ICLR] AnyBCQ: Hardware Efficient Flexible Binary-Coded Quantization for Multi-Precision LLMs [code]
• [ICLR] Tequila: Deadzone-free Ternary Quantization for Large Language Models
• [ICLR] LogART: Pushing the Limit of Efficient Logarithmic Post-Training Quantization [code]
• [ICLR] ParoQuant: Pairwise Rotation Quantization for Efficient Reasoning LLM Inference [code]
• [ICLR] Improving Block-Wise LLM Quantization by 4-bit Generalized Normal Float Formats
• [ICLR] Channel-Aware Mixed-Precision Quantization for Efficient Long-Context Inference
• [ICLR] CodeQuant: Unified Clustering and Quantization for Enhanced Outlier Smoothing in Low-Precision Mixture-of-Experts
• [ICLR] QeRL: Beyond Efficiency - Quantization-enhanced Reinforcement Learning for LLMs [code]
• [ICLR] AutoQVLA: Not All Channels Are Equal in Vision-Language-Action Model's Quantization
• [ICLR] Achieving low-bit Muon through subspace preservation and grid quantization
• [ICLR] Shift-and-Sum Quantization for Visual Autoregressive Models
• [ICLR] Inlier-Centric Post-Training Quantization for Object Detection Models
• [ICLR] Efficient Quantization of Mixture-of-Experts with Theoretical Generalization Guarantees
• [ICLR] BBQ: Boosting Quantization Entropy with Bell Box Quantization
• [ICLR] Improving Block-Wise LLM Quantization by 4-bit Block-Wise Optimal Float (BOF4): Analysis and Variations [code]
• [ICLR] Learning under Quantization for High-Dimensional Linear Regression
• [ICLR] On-the-Fly Adaptation to Quantization: Configuration-Aware LoRA for Efficient Fine-Tuning of Quantized LLMs
• [ICLR] Bridging the Gap Between Promise and Performance for FP4 Quantization [code]
• [ICLR] KBVQ-MoE: KLT-guided SVD with Bias-Corrected Vector Quantization for MoE Large Language Models [code]
• [ICLR] UniQL: Unified Quantization and Low-rank Compression for Adaptive Edge LLMs [code]
• [ICLR] The Lattice Geometry of Neural Network Quantization: A Short Equivalence Proof of GPTQ and Babai's algorithm
• [ICLR] DPQuant: Efficient and Private Model Training via Dynamic Quantization Scheduling
• [ICLR] Towards Quantization-Aware Training for Ultra-Low-Bit Reasoning LLMs
• [ICLR] A Convergence Analysis of Adaptive Optimizers under Floating-point Quantization
• [ICLR] Training Dynamics Impact Post-Training Quantization Robustness [code]
• [ICLR] SSDi8: Accurate and Efficient 8-bit Quantization for State Space Duality
• [ICLR] The Geometry of LLM Quantization: GPTQ as Babai's Nearest Plane Algorithm
• [ICLR] PTQ4ARVG: Post-Training Quantization for AutoRegressive Visual Generation Models [code]
• [ICLR] QWHA: Quantization-Aware Walsh-Hadamard Adaptation for Parameter-Efficient Fine-Tuning on Large Language Models [code]
• [ICLR] Gradient-Aligned Calibration for Post-Training Quantization of Diffusion Models
• [ICLR] SERQ: Saliency-Aware Low-Rank Error Reconstruction for LLM Quantization
• [ICLR] Compute-Optimal Quantization-Aware Training
• [ICLR] PM-KVQ: Progressive Mixed-precision KV Cache Quantization for Long-CoT LLMs [code]
• [ICLR] Beyond Outliers: A Study of Optimizers Under Quantization
• [ICLR] Qronos: Correcting the Past by Shaping the Future... in Post-Training Quantization
• [ICLR] MicroMix: Efficient Mixed-Precision Quantization with Microscaling Formats for Large Language Models [code]
• [ICLR] TurboBoA: Faster and Exact Attention-aware Quantization without Backpropagation
• [ICLR] Beyond Uniformity: Sample and Frequency Meta Weighting for Post-Training Quantization of Diffusion Models
• [ICLR] Rethinking Residual Errors in Compensation-based LLM Quantization
• [ICLR] SPR²Q: Static Priority-based Rectifier Routing Quantization for Image Super-Resolution [code]
• [ICLR] STaMP: Sequence Transformation and Mixed Precision for Low-Precision Activation Quantization
• [arXiv] Sherry: Hardware-Efficient 1.25-Bit Ternary Quantization via Fine-grained Sparsification [code]
• [arXiv] D²Quant: Accurate Low-bit Post-Training Weight Quantization for LLMs
• [arXiv] QuantLRM: Quantization of Large Reasoning Models via Fine-Tuning Signals
• [arXiv] SliderQuant: Accurate Post-Training Quantization for LLMs
• [arXiv] What Makes Low-Bit Quantization-Aware Training Work for Reasoning LLMs? A Systematic Study
• [arXiv] OneComp: One-Line Revolution for Generative AI Model Compression [Code]

2025

• [ICML] Q-VDiT: Towards Accurate Quantization and Distillation of Video-Generation Diffusion Transformers [code]
• [AAAI] MPQ-DM: Mixed Precision Quantization for Extremely Low Bit Diffusion Models
• [ICML] SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Models [code]
• [TPAMI] BiVM: Accurate Binarized Neural Network for Efficient Video Matting
• [NeurIPS] S²Q-VDiT: Accurate Quantized Video Diffusion Transformer with Salient Data and Sparse Token Distillation [code]
• [CVPR] PassionSR: Post-Training Quantization with Adaptive Scale in One-Step Diffusion based Image Super-Resolution [code]
• [ICLR] ARB-LLM: Alternating Refined Binarizations for Large Language Models [code]
• [ICLR] BinaryDM: Accurate Weight Binarization for Efficient Diffusion Models [code]
• [ICML] FlatQuant: Flatness Matters for LLM Quantization [code]
• [ICML] RoSTE: An Efficient Quantization-Aware Supervised Fine-Tuning Approach for Large Language Models [code]
• [ICML] GANQ: GPU-Adaptive Non-Uniform Quantization for Large Language Models
• [ICML] Modulated Diffusion: Accelerating Generative Modeling with Modulated Quantization [code]
• [NeurIPS] DartQuant: Efficient Rotational Distribution Calibration for LLM Quantization [code]
• [AAAI] JAQ: Joint Efficient Architecture Design and Low-Bit Quantization
• [AAAI] OAC: Output-adaptive Calibration for Accurate Post-Training Quantization of LLMs
• [AAAI] Optimizing Quantized Diffusion Models via Distillation with Decay Timestep-Aware Loss
• [AAAI] Quantifiable Quantization Sensitivity of Diffusion Models
• [AAAI] TCAQ-DM: Timestep-Channel Adaptive Quantization for Diffusion Models
• [ACL] EfficientQAT: Efficient Quantization-Aware Training for Large Language Models [code]
• [ACL] L4Q: Parameter Efficient Quantization-Aware Fine-Tuning on Large Language Models
• [ACL] MoQAE: Mixed-Precision Quantization for Long-Context LLM Inference via Mixture of Quantization-Aware Experts
• [ACL] Outlier-Safe Pre-Training for Robust 4-Bit Quantization of Large Language Models
• [ACL] PTQ1.61: Push the Real Limit of Extremely Low-Bit Post-Training Quantization Methods for Large Language Models [code]
• [ACL] Unifying Uniform and Binary-coding Quantization for Accurate Compression of Large Language Models
• [ACL] “Give Me BF16 or Give Me Death”? Accuracy-Performance Trade-Offs in LLM Quantization
• [ACM MM] DilateQuant: Accurate and Efficient Quantization-Aware Training for Diffusion Models via Weight Dilation
• [ACM MM] Learning Binarized Representations with Pseudo-positive Distillation
• [ACM MM] MQuant: Unleashing the Inference Potential of Multimodal Large Language Models with Post-Training Quantization
• [ACM MM] Pushing the Limit of Binarized Neural Network for Image Super Resolution with Smooth Information Transmission
• [ACM MM] Quantization Meets OOD: Generalizable Quantization-aware Training from a Flatness Perspective
• [EMNLP] AMQ: Enabling AutoML for Mixed-precision Weight-Only Quantization of Large Language Models
• [EMNLP] Does quantization affect models' performance on long-input and long-output tasks?
• [ICLR] CBQ: Cross-Block Quantization for Large Language Models
• [ICLR] DGQ: Distribution-Aware Group Quantization for Text-to-Image Diffusion Models
• [ICLR] LeanQuant: Accurate and Scalable Large Language Model Quantization with Loss-error-aware Grid
• [ICLR] OSTQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fitting [code]
• [ICLR] QERA: an Analytical Framework for Quantization Error Reconstruction [code]
• [ICLR] SpinQuant: LLM Quantization with Learned Rotations
• [ICLR] SVDQuant: Absorbing Outliers by Low-Rank Component for 4-Bit Diffusion Models
• [ICLR] ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation
• [ICML] GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidance [code]
• [ICML] ResQ: Mixed-Precision Quantization of Large Language Models with Low-Rank Residuals [code]
• [NeurIPS] A Double Normalization Approach for Calibration-Free Low-Bit KV Cache Quantization
• [NeurIPS] Binary Quadratic Quantization: Beyond First-Order Quantization for Real-Valued Matrix Compression
• [NeurIPS] Learning Grouped Lattice Vector Quantizers for Low-Bit Large Language Models
• [NeurIPS] LittleBit: Ultra Low-Bit Quantization via Latent Factorization
• [NeurIPS] ParetoQ: Improving Scaling Laws in Extremely Low-bit LLM Quantization
• [NeurIPS] Q-Palette: Fractional-Bit Quantizers Toward Optimal Weight-Only Post-Training Quantization
• [NeurIPS] Wavelet-Enhanced High-Fidelity 1-Bit Quantization for LLMs
• [NeurIPS] Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantization [Code]
• [ACL Findings] Achieving Binary Weight and Activation for LLMs using Post-Training Quantization
• [EMNLP Findings] KurTail: Kurtosis-based LLM Quantization
• [SIGMOD] Practical and Asymptotically Optimal Quantization of High-Dimensional Vectors in Euclidean Space for Approximate Nearest Neighbor Search [code]
• [NeurIPS] QBasicVSR: Temporal Awareness Adaptation Quantization for Video Super-Resolution
• [NeurIPS] Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantization
• [NeurIPS] Point4Bit: Post Training 4-bit Quantization for Point Cloud 3D Detection
• [NeurIPS] PMQ-VE: Progressive Multi-Frame Quantization for Video Enhancement [code]
• [NeurIPS] VETA-DiT: Variance-Equalized and Temporally Adaptive Quantization for Efficient 4-bit Diffusion Transformers
• [NeurIPS] LoTA-QAF: Lossless Ternary Adaptation for Quantization-Aware Fine-Tuning [code]
• [NeurIPS] Efficient Multi-bit Quantization Network Training via Weight Bias Correction and Bit-wise Coreset Sampling
• [NeurIPS] Efficient and Generalizable Mixed-Precision Quantization via Topological Entropy
• [NeurIPS] QSCA: Quantization with Self-Compensating Auxiliary for Monocular Depth Estimation
• [ICCV] Scheduling Weight Transitions for Quantization-Aware Training [code]
• [ICCV] Task-Specific Zero-shot Quantization-Aware Training for Object Detection [code]
• [ICCV] OuroMamba: A Data-Free Quantization Framework for Vision Mamba
• [ICCV] FedWSQ: Efficient Federated Learning with Weight Standardization and Distribution-Aware Non-Uniform Quantization [code]
• [ICCV] Semantic Alignment and Reinforcement for Data-Free Quantization of Vision Transformers [code]
• [ICCV] QuantCache: Adaptive Importance-Guided Quantization with Hierarchical Latent and Layer Caching for Video Generation [code]
• [ICCV] MixA-Q: Revisiting Activation Sparsity for Vision Transformers from a Mixed-Precision Quantization Perspective
• [ICCV] DMQ: Dissecting Outliers of Diffusion Models for Post-Training Quantization [code]
• [ICCV] AHCPTQ: Accurate and Hardware-Compatible Post-Training Quantization for Segment Anything Model
• [ICCV] MSQ: Memory-Efficient Bit Sparsification Quantization
• [ICCV] QuEST: Low-bit Diffusion Model Quantization via Efficient Selective Finetuning [code]
• [ICML] MxMoE: Mixed-precision Quantization for MoE with Accuracy and Performance Co-Design [code]
• [ICML] Learning from Loss Landscape: Generalizable Mixed-Precision Quantization via Adaptive Sharpness-Aware Gradient Aligning
• [ICML] PARQ: Piecewise-Affine Regularized Quantization [code]
• [ICML] Quamba2: A Robust and Scalable Post-training Quantization Framework for Selective State Space Models [code]
• [ICML] LRA-QViT: Integrating Low-Rank Approximation and Quantization for Robust and Efficient Vision Transformers
• [ICML] BoA: Attention-aware Post-training Quantization without Backpropagation
• [ICML] MoEQuant: Enhancing Quantization for Mixture-of-Experts Large Language Models via Expert-Balanced Sampling and Affinity Guidance [code]
• [ICML] NestQuant: nested lattice quantization for matrix products and LLMs
• [ICML] Q-resafe: Assessing Safety Risks and Quantization-aware Safety Patching for Quantized Large Language Models [code]
• [ICML] SLiM: One-shot Quantization and Sparsity with Low-rank Approximation for LLM Weight Compression [code]
• [ICML] QT-DoG: Quantization-Aware Training for Domain Generalization [code]
• [ICML] Matryoshka Quantization
• [ICML] Merge-Friendly Post-Training Quantization for Multi-Target Domain Adaptation [code]
• [ICML] Layer-wise Quantization for Quantized Optimistic Dual Averaging
• [ICML] Outlier-Aware Post-Training Quantization for Discrete Graph Diffusion Models
• [ICML] BlockDialect: Block-wise Fine-grained Mixed Format Quantization for Energy-Efficient LLM Inference
• [ICML] GPTAQ: Efficient Finetuning-Free Quantization with Asymmetric Calibration [code]
• [ICML] Optimizing Large Language Model Training Using FP4 Quantization
• [ICML] SKIM: Any-bit Quantization Pushing The Limits of Post-Training Quantization
• [ICML] SageAttention2: Efficient Attention with Thorough Outlier Smoothing and Per-thread INT4 Quantization [code]
• [AAAI] Thinking in Granularity: Dynamic Quantization for Image Super-Resolution by Intriguing Multi-Granularity Clues [code]
• [AAAI] D2-DPM: Dual Denoising for Quantized Diffusion Probabilistic Models [code]
• [CVPR] Quantization without Tears
• [CVPR] APHQ-ViT: Post-Training Quantization with Average Perturbation Hessian Based Reconstruction for Vision Transformer [code]
• [ICLR] SynQ: Accurate Zero-shot Quantization by Synthesis-aware Fine-tuning [code]

2024

• [ICML] BiLLM: Pushing the Limit of Post-Training Quantization for LLMs [code]
• [ICML] Compressing Large Language Models by Joint Sparsification and Quantization
• [NeurIPS] BiDM: Pushing the Limit of Quantization for Diffusion Models
• [ACL Findings] DB-LLM: Accurate Dual-Binarization for Efficient LLMs
• [NeurIPS] Binarized Diffusion Model for Image Super-Resolution [code]
• [NeurIPS] 2DQuant: Low-bit Post-Training Quantization for Image Super-Resolution [code]
• [ICML] Accurate LoRA-Finetuning Quantization of LLMs via Information Retention [code]
• [ICML] Flexible Residual Binarization for Image Super-Resolution
• [AAAI] Agile-Quant: Activation-Guided Quantization for Faster Inference of LLMs on the Edge
• [AAAI] AQ-DETR: Low-Bit Quantized Detection Transformer with Auxiliary Queries
• [AAAI] Bi-ViT: Pushing the Limit of Vision Transformer Quantization
• [AAAI] Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensation
• [AAAI] Make RepVGG Greater Again: A Quantization-Aware Approach
• [AAAI] MetaMix: Meta-State Precision Searcher for Mixed-Precision Activation Quantization
• [AAAI] Norm Tweaking: High-Performance Low-Bit Quantization of Large Language Models
• [AAAI] OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Models
• [AAAI] PTMQ: Post-training Multi-Bit Quantization of Neural Networks
• [AAAI] Robustness-Guided Image Synthesis for Data-Free Quantization
• [AAAI] What Makes Quantization for Large Language Model Hard? An Empirical Study from the Lens of Perturbation
• [ACL] Improving Conversational Abilities of Quantized Large Language Models via Direct Preference Alignment
• [ACM MM] Advancing Multimodal Large Language Models with Quantization-Aware Scale Learning Based on Warmup
• [CVPR] Data-Free Quantization via Pseudo-label Filtering
• [CVPR] Enhancing Post-training Quantization Calibration through Contrastive Learning
• [CVPR] Instance-Aware Group Quantization for Vision Transformers
• [CVPR] Mixed-Precision Quantization for Federated Learning on Resource-Constrained Heterogeneous Devices
• [CVPR] PTQ4SAM: Post-Training Quantization for Segment Anything
• [CVPR] Reg-PTQ: Regression-specialized Post-training Quantization for Fully Quantized Object Detector
• [CVPR] Retraining-Free Model Quantization via One-Shot Weight-Coupling Learning
• [CVPR] TFMQ-DM: Temporal Feature Maintenance Quantization for Diffusion Models
• [CVPR] Towards Accurate Post-training Quantization for Diffusion Models
• [ECCV] AdaLog: Post-Training Quantization for Vision Transformers with Adaptive Logarithm Quantizer
• [ECCV] CLAMP-ViT: Contrastive Data-Free Learning for Adaptive Post-Training Quantization of ViTs
• [ECCV] Memory-Efficient Fine-Tuning for Quantized Diffusion Model
• [ECCV] MetaAug: Meta-Data Augmentation for Post-Training Quantization
• [ECCV] MixDQ: Memory-Efficient Few-Step Text-to-Image Diffusion Models with Metric-Decoupled Mixed Precision Quantization
• [ECCV] Overcoming Distribution Mismatch in Quantizing Image Super-Resolution Networks
• [ECCV] Post-training Quantization with Progressive Calibration and Activation Relaxing for Text-to-Image Diffusion Models
• [ECCV] PQ-SAM: Post-training Quantization for Segment Anything Model
• [ECCV] Timestep-Aware Correction for Quantized Diffusion Models
• [ECCV] Towards Robust Full Low-bit Quantization of Super Resolution Networks
• [EMNLP] ApiQ: Finetuning of 2-Bit Quantized Large Language Model
• [EMNLP] Prefixing Attention Sinks can Mitigate Activation Outliers for Large Language Model Quantization
• [EMNLP] VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Models
• [ICLR] AffineQuant: Affine Transformation Quantization for Large Language Models [code]
• [ICLR] EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models
• [ICLR] LiDAR-PTQ: Post-Training Quantization for Point Cloud 3D Object Detection
• [ICLR] LoftQ: LoRA-Fine-Tuning-aware Quantization for Large Language Models [code]
• [ICLR] LUT-GEMM: Quantized Matrix Multiplication based on LUTs for Efficient Inference in Large-Scale Generative Language Models
• [ICLR] OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models [code]
• [ICLR] PB-LLM: Partially Binarized Large Language Models [code]
• [ICLR] QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Models [code]
• [ICLR] QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models
• [ICLR] Rethinking Channel Dimensions to Isolate Outliers for Low-bit Weight Quantization of Large Language Models
• [ICLR] SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression [code]
• [ICML] A2Q+: Improving Accumulator-Aware Weight Quantization
• [ICML] BiE: Bi-Exponent Block Floating-Point for Large Language Models Quantization
• [ICML] ERQ: Error Reduction for Post-Training Quantization of Vision Transformers
• [ICML] Evaluating Quantized Large Language Models
• [ICML] Extreme Compression of Large Language Models via Additive Quantization
• [ICML] FrameQuant: Flexible Low-Bit Quantization for Transformers
• [ICML] KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache [code]
• [ICML] LQER: Low-Rank Quantization Error Reconstruction for LLMs
• [ICML] Outlier-aware Slicing for Post-Training Quantization in Vision Transformer
• [ICML] Sharpness-Aware Data Generation for Zero-shot Quantization
• [ICML] SqueezeLLM: Dense-and-Sparse Quantization [code]
• [MLSys] AWQ: Activation-aware Weight Quantization for On-Device LLM Compression and Acceleration [code]
• [NeurIPS] BitsFusion: 1.99 bits Weight Quantization of Diffusion Model
• [NeurIPS] DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMs
• [NeurIPS] KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantization
• [NeurIPS] KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization
• [NeurIPS] PTQ4DiT: Post-training Quantization for Diffusion Transformers
• [NeurIPS] Q-VLM: Post-training Quantization for Large Vision-Language Models
• [NeurIPS] QBB: Quantization with Binary Bases for LLMs
• [NeurIPS] ZipCache: Accurate and Efficient KV Cache Quantization with Salient Token Identification
• [ACL Findings] A Comprehensive Evaluation of Quantization Strategies for Large Language Models
• [ACL Findings] AFPQ: Asymmetric Floating Point Quantization for LLMs [code]
• [ACL Findings] LLM-QAT: Data-Free Quantization Aware Training for Large Language Models
• [EMNLP Findings] ATQ: Activation Transformation for Weight-Activation Quantization of LLMs
• [EMNLP Findings] Fine-tuning Rotated Outlier-free LLMs for Effective Weight-Activation Quantization
• [EMNLP Findings] How Does Quantization Affect Multilingual LLMs?
• [EMNLP Findings] MobileQuant: Mobile-friendly Quantization for On-device Language Models
• [EMNLP Findings] QEFT: Quantization for Efficient Fine-Tuning of LLMs
• [EMNLP Industry] LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit
• [arXiv] APTQ: Attention-aware Post-Training Mixed-Precision Quantization for Large Language Models
• [arXiv] EasyQuant: An Efficient Data-free Quantization Algorithm for LLMs
• [arXiv] EdgeQAT: Entropy and Distribution Guided Quantization-Aware Training for the Acceleration of Lightweight LLMs on the Edge [code]
• [arXiv] FlattenQuant: Breaking Through the Inference Compute-bound for Large Language Models with Per-tensor Quantization
• [arXiv] GPTVQ: The Blessing of Dimensionality for LLM Quantization [code]
• [arXiv] IntactKV: Improving Large Language Model Quantization by Keeping Pivot Tokens Intact
• [arXiv] OneBit: Towards Extremely Low-bit Large Language Models
• [arXiv] QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs [code]
• [arXiv] RepQuant: Towards Accurate Post-Training Quantization of Large Transformer Models via Scale Reparameterization
• [SIGMOD] RaBitQ: Quantizing High-Dimensional Vectors with a Theoretical Error Bound for Approximate Nearest Neighbor Search [code]
• [AAAI] One-Step Forward and Backtrack: Overcoming Zig-Zagging in Loss-Aware Quantization Training
• [ICML] Learning from students: Applying t-distributions to explore accurate and efficient formats for llms [code]
• [ICML] Jetfire: Efficient and Accurate Transformer Pretraining with INT8 Data Flow and Per-Block Quantization
• [ICML] Reshape and Adapt for Output Quantization (RAOQ): Quantization-aware Training for In-memory Computing Systems
• [ICML] QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks [code]
• [NeurIPS] Towards Next-Level Post-Training Quantization of Hyper-Scale Transformers [code]
• [NeurIPS] MagR: Weight Magnitude Reduction for Enhancing Post-Training Quantization [code]
• [NeurIPS] Exploiting LLM Quantization
• [NeurIPS] Efficient Multi-task LLM Quantization and Serving for Multiple LoRA Adapters
• [NeurIPS] QTIP: Quantization with Trellises and Incoherence Processing [code]
• [NeurIPS] Generalizing CNNs to graphs with learnable neighborhood quantization [code]