CUSTOM KNOWLEDGE FEED

#quantization

9 cards

This feed is generated directly from exact card hashtags; there is no separate feed-content copy.

Subscribe to this viewRSSJSON

Last verified: 2026-07-14 — refresh when CUDA, PyTorch, or the DGX Spark stack ships a new major version. Use it to give an agent explicit responsibilities, steps and constraints.

llm-engineering/inference-and-quantizationguide#llm-ml-engineering #inference-and-quantization #cuda #abi

#llm-engineering · by Collider.club

★ 0◌ 0

AQLM

Official PyTorch implementation for Extreme Compression of Large Language Models via Additive Quantization. Use it to navigate the topic and choose relevant methods, papers or tools.

llm-engineering/inference-and-quantizationreference#llm-ml-engineering #inference-and-quantization #aqlm #models

#llm-engineering · by Collider.club

★ 0◌ 0

Awesome Model Quantization

This repo collects papers, documents, and codes about model quantization for anyone who wants to research it. We are continuously improving the project. Use it to navigate the topic and choose relevant methods, papers or tools.

llm-engineering/inference-and-quantizationreference#llm-ml-engineering #inference-and-quantization #papers #model

#llm-engineering · by Collider.club

★ 0◌ 0

Lit-LLaMAAgent

⚠️ Warning: Not Actively Maintained This repository is no longer actively maintained. For a more up-to-date alternative, please visit the LitGPT project: https://github.com/Lightning-AI/litgpt , which serves as the successor to this repository.

llm-engineering/serving-and-retrievalguide#python #models #llama #quantization

#llm-engineering · by MDRSS Source Library Github collector

★ 0◌ 0

rwkv.cppAgent

This is a port of BlinkDL/RWKV-LM to ggerganov/ggml. Besides the usual FP32, it supports FP16, quantized INT4, INT5 and INT8 inference.

llm-engineering/serving-and-retrievalguide#deep-learning #ggml #language-model #llm

#llm-engineering · by MDRSS Source Library Github collector

★ 0◌ 0

Faster Whisper transcription with CTranslate2Agent

faster-whisper is a reimplementation of OpenAI's Whisper model using CTranslate2, which is a fast inference engine for Transformer models. This implementation is up to 4 times faster than openai/whisper for the same accuracy while using less memory.

multimodal/speech-and-audioguide#deep-learning #inference #openai #quantization

#multimodal · by MDRSS Source Library Github collector

★ 0◌ 0

QLoRA: Efficient Finetuning of Quantized LLMsAgent

This repo supports the paper "QLoRA: Efficient Finetuning of Quantized LLMs", an effort to democratize access to LLM research. QLoRA uses bitsandbytes for quantization and is integrated with Hugging Face's PEFT and transformers libraries.

llm-engineering/serving-and-retrievalguide#models #llms #quantization

#llm-engineering · by MDRSS Source Library Github collector

★ 0◌ 0

LLM CourseAgent

𝕏 Follow me on X • 🤗 Hugging Face • 💻 Blog • 📙 LLM Engineer's Handbook The LLM course is divided into three parts: 1. 🧩 LLM Fundamentals is optional and covers fundamental knowledge about mathematics, Python, and neural networks.

llm-engineering/serving-and-retrievalreference#course #large-language-models #llm #machine-learning

#llm-engineering · by MDRSS Source Library Github collector

★ 0◌ 0

Core ML Stable DiffusionAgent

Run Stable Diffusion on Apple Silicon with Core ML [\[Blog Post\]](https://machinelearning.apple.com/research/stable-diffusion-coreml-apple-silicon) [\[BibTeX\]](#bibtex) This repository comprises: If you run into issues during installation or runtime, please refer to the FAQ section. Please refer to the System Requirements section before getting started.

llm-engineering/serving-and-retrievalguide#python #models #diffusion #benchmarks

#llm-engineering · by MDRSS Source Library Github collector

★ 0◌ 0

CUSTOM KNOWLEDGE FEED

#quantization

9 cards

This feed is generated directly from exact card hashtags; there is no separate feed-content copy.

Subscribe to this viewRSSJSON

G1: CUDA 12/13 ABI Mismatch

Last verified: 2026-07-14 — refresh when CUDA, PyTorch, or the DGX Spark stack ships a new major version. Use it to give an agent explicit responsibilities, steps and constraints.

llm-engineering/inference-and-quantizationguide#llm-ml-engineering #inference-and-quantization #cuda #abi

#llm-engineering · by Collider.club

★ 0◌ 0

AQLM

Official PyTorch implementation for Extreme Compression of Large Language Models via Additive Quantization. Use it to navigate the topic and choose relevant methods, papers or tools.

llm-engineering/inference-and-quantizationreference#llm-ml-engineering #inference-and-quantization #aqlm #models

#llm-engineering · by Collider.club

★ 0◌ 0

Awesome Model Quantization

llm-engineering/inference-and-quantizationreference#llm-ml-engineering #inference-and-quantization #papers #model

#llm-engineering · by Collider.club

★ 0◌ 0

Lit-LLaMAAgent

llm-engineering/serving-and-retrievalguide#python #models #llama #quantization

#llm-engineering · by MDRSS Source Library Github collector

★ 0◌ 0

rwkv.cppAgent