# LLM Engineering — MDRSS semantic catalog

> Semantic domain: llm-engineering
> Model training, serving, retrieval, evaluation, efficiency, alignment, and infrastructure.
> Filters: category=serving-and-retrieval
> Aggregate catalog URL: https://mdrss.com/catalog/llm-engineering?category=serving-and-retrieval

## Feeds

- [Data, Research & Open Knowledge](https://mdrss.com/s/data-research-and-open-knowledge) — Datasets, reproducible research, FAIR data, open knowledge, and analysis methods.
- [Software & Open Documentation](https://mdrss.com/s/software-and-open-documentation) — Open technical documentation, developer platforms, infrastructure, and tooling.

## Aggregate endpoints

- RSS: https://mdrss.com/catalog/llm-engineering/rss.xml?category=serving-and-retrieval
- JSON: https://mdrss.com/catalog/llm-engineering/feed.json?category=serving-and-retrieval

## Semantic domain cards (19)

### [Chat with PDF locally using Ollama + LangChain](https://mdrss.com/llm-engineering/serving-and-retrieval/2309/2309.md)

A powerful local RAG (Retrieval Augmented Generation) application that lets you chat with your PDF documents using Ollama and LangChain. This project includes multiple interfaces: a modern Next.js web app, a Streamlit interface, and Jupyter notebooks for experimentation.

Feed: [llm-engineering](https://mdrss.com/s/llm-engineering) · Snapshot: 2026-08-04T12:18:33.634Z · Version: 1

### [Lit-LLaMA](https://mdrss.com/llm-engineering/serving-and-retrieval/2164/2164.md)

⚠️ Warning: Not Actively Maintained This repository is no longer actively maintained. For a more up-to-date alternative, please visit the LitGPT project: https://github.com/Lightning-AI/litgpt , which serves as the successor to this repository.

Feed: [llm-engineering](https://mdrss.com/s/llm-engineering) · Snapshot: 2026-08-04T12:18:49.845Z · Version: 1

### [Text Generation Inference](https://mdrss.com/llm-engineering/serving-and-retrieval/2140/2140.md)

A Rust, Python and gRPC server for text generation inference. Used in production at Hugging Face to power Hugging Chat, the Inference API and Inference Endpoints.

Feed: [llm-engineering](https://mdrss.com/s/llm-engineering) · Snapshot: 2026-08-04T12:18:48.518Z · Version: 1

### [rwkv.cpp](https://mdrss.com/llm-engineering/serving-and-retrieval/2075/2075.md)

This is a port of BlinkDL/RWKV-LM to ggerganov/ggml. Besides the usual FP32, it supports FP16, quantized INT4, INT5 and INT8 inference.

Feed: [llm-engineering](https://mdrss.com/s/llm-engineering) · Snapshot: 2026-08-04T12:18:33.634Z · Version: 1

### [koboldcpp](https://mdrss.com/llm-engineering/serving-and-retrieval/1899/1899.md)

KoboldCpp is an easy-to-use AI text-generation software for GGML and GGUF models, inspired by the original KoboldAI. It's a single self-contained distributable that builds off llama.cpp and adds many additional powerful features.

Feed: [llm-engineering](https://mdrss.com/s/llm-engineering) · Snapshot: 2026-08-04T12:18:48.518Z · Version: 1

### [DLLM RL](https://mdrss.com/llm-engineering/serving-and-retrieval/1891/1891.md)

We also introduce a diffusion-based value model that reduces variance and improves stability during optimization. Based on TraceRL, we derive a series of diffusion language models, TraDo, which achieve state-of-the-art performance on math and coding reasoning tasks.

Feed: [llm-engineering](https://mdrss.com/s/llm-engineering) · Snapshot: 2026-08-04T12:18:33.634Z · Version: 1

### [REST API examples](https://mdrss.com/llm-engineering/serving-and-retrieval/1726/1726.md)

I've started to work on reimplementation of the library here: FastTensors Please star it if you'd like to see GGML-compatible implementation in pure Go. Please check out my related project Booster We dream of a world where fellow ML hackers are grokking REALLY BIG GPT models in their homelabs without having GPU clusters consuming a shit tons of $$$.

Feed: [llm-engineering](https://mdrss.com/s/llm-engineering) · Snapshot: 2026-08-04T12:18:33.634Z · Version: 1

### [QLoRA: Efficient Finetuning of Quantized LLMs](https://mdrss.com/llm-engineering/serving-and-retrieval/1671/1671.md)

This repo supports the paper "QLoRA: Efficient Finetuning of Quantized LLMs", an effort to democratize access to LLM research. QLoRA uses bitsandbytes for quantization and is integrated with Hugging Face's PEFT and transformers libraries.

Feed: [llm-engineering](https://mdrss.com/s/llm-engineering) · Snapshot: 2026-08-04T12:18:33.634Z · Version: 1

### [AutoRAG](https://mdrss.com/llm-engineering/serving-and-retrieval/1593/1593.md)

A self-evolving librarian agent for document collections. AutoRAG searches your PDFs, wikis, notes, research papers, and knowledge bases — then curates the results into clean, numbered knowledge units.

Feed: [llm-engineering](https://mdrss.com/s/llm-engineering) · Snapshot: 2026-08-04T12:18:32.550Z · Version: 1

### [LLM Zoo: democratizing ChatGPT](https://mdrss.com/llm-engineering/serving-and-retrieval/1511/1511.md)

⚡LLM Zoo is a project that provides data, models, and evaluation benchmark for large language models.⚡  Tech Report (assets/llmzoo.pdf) technology gifted by the creator. For example, many pioneers have made great efforts to spread the use of light bulbs and vaccines to developing countries.

Feed: [llm-engineering](https://mdrss.com/s/llm-engineering) · Snapshot: 2026-08-04T12:18:48.518Z · Version: 1

### [What you'll learn](https://mdrss.com/llm-engineering/serving-and-retrieval/1425/1425.md)

LLM Twin Course: Building Your Production-Ready AI Replica Learn to architect and implement a production-ready LLM & RAG system by building your LLM Twin From data gathering to productionizing LLMs using LLMOps good practices. by Decoding AI By finishing the "LLM Twin: Building Your Production-Ready AI Replica" free course, you will learn how to design, train, and deploy a production-ready LLM twin of yourself powered by LLMs, vector DBs, and LLMOps good practices.

Feed: [llm-engineering](https://mdrss.com/s/llm-engineering) · Snapshot: 2026-08-04T12:18:48.518Z · Version: 1

### [Comparison with FAISS](https://mdrss.com/llm-engineering/serving-and-retrieval/1383/1383.md)

USearch and FAISS both employ the same HNSW algorithm, but they differ significantly in their design principles. USearch is compact and broadly compatible without sacrificing performance, primarily focusing on user-defined metrics and fewer dependencies.

Feed: [llm-engineering](https://mdrss.com/s/llm-engineering) · Snapshot: 2026-08-04T12:18:32.550Z · Version: 1

### [Evalscope](https://mdrss.com/llm-engineering/serving-and-retrieval/1280/1280.md)

中文 &nbsp ｜ &nbsp English &nbsp 📖 中文文档 &nbsp ｜ &nbsp 📖 English Documentation EvalScope is a one-stop LLM evaluation framework built by the ModelScope Community. Just one command to start — it supports model capability evaluation, inference performance stress testing, and result visualization.

Feed: [llm-engineering](https://mdrss.com/s/llm-engineering) · Snapshot: 2026-08-04T12:18:33.634Z · Version: 1

### [PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU](https://mdrss.com/llm-engineering/serving-and-retrieval/1244/1244.md)

PowerInfer is a CPU/GPU LLM inference engine leveraging activation locality for your device. Project Kanban https://github.com/SJTU-IPADS/PowerInfer/assets/34213478/fe441a42-5fce-448b-a3e5-ea4abb43ba23 PowerInfer v.s.

Feed: [llm-engineering](https://mdrss.com/s/llm-engineering) · Snapshot: 2026-08-04T12:18:32.550Z · Version: 1

### [Ollama Swift Client](https://mdrss.com/llm-engineering/serving-and-retrieval/1183/1183.md)

A Swift client library for interacting with the Ollama API. Pass "json" to get back a JSON string, or specify a full JSON Schema: The format parameter works with both chat and generate methods.

Feed: [llm-engineering](https://mdrss.com/s/llm-engineering) · Snapshot: 2026-08-04T12:18:48.518Z · Version: 1

### [LLM Course](https://mdrss.com/llm-engineering/serving-and-retrieval/1089/1089.md)

𝕏 Follow me on X • 🤗 Hugging Face • 💻 Blog • 📙 LLM Engineer's Handbook The LLM course is divided into three parts: 1. 🧩 LLM Fundamentals is optional and covers fundamental knowledge about mathematics, Python, and neural networks.

Feed: [llm-engineering](https://mdrss.com/s/llm-engineering) · Snapshot: 2026-08-04T12:18:32.550Z · Version: 1

### [Get started](https://mdrss.com/llm-engineering/serving-and-retrieval/1049/1049.md)

Unsloth Studio lets you run and train models locally. Features • News • Quickstart • Notebooks • Documentation Unsloth Studio (Beta) lets you run and train text, audio, embedding, vision models on Windows, Linux and macOS.

Feed: [llm-engineering](https://mdrss.com/s/llm-engineering) · Snapshot: 2026-08-04T12:18:32.550Z · Version: 1

### [ODS](https://mdrss.com/llm-engineering/serving-and-retrieval/1025/1025.md)

Osmantic Deployment System Turn your PC, Mac, or Linux box into a private AI server. AI server and homelab setup is rapidly becoming a solved problem.

Feed: [llm-engineering](https://mdrss.com/s/llm-engineering) · Snapshot: 2026-08-04T12:18:48.518Z · Version: 1

### [Core ML Stable Diffusion](https://mdrss.com/llm-engineering/serving-and-retrieval/938/938.md)

Run Stable Diffusion on Apple Silicon with Core ML  \ Blog Post\ (https://machinelearning.apple.com/research/stable-diffusion-coreml-apple-silicon)  \ BibTeX\ (#bibtex) This repository comprises: If you run into issues during installation or runtime, please refer to the FAQ section. Please refer to the System Requirements section before getting started.

Feed: [llm-engineering](https://mdrss.com/s/llm-engineering) · Snapshot: 2026-08-04T12:18:32.550Z · Version: 1