LLM (Large Language Models) FineTuning Projects and notes on common practical techniques

Snapshot 2026-08-04 13:48:05 UTC · version 1

published
INDEXABLE MARKDOWN SNAPSHOT

Research document

Open canonical .md

LLM (Large Language Models) FineTuning Projects and notes on common practical techniques

logo]: https://github.com/rohan-paul/rohan-paul/blob/master/assets/png. Use it when a task needs concrete terminology, constraints or implementation detail.

Editorial note: curated source snapshot published by Collider.club under the MIT License. Source attribution is preserved in the front matter.

Source snapshot

LLM (Large Language Models) FineTuning Projects and notes on common practical techniques

Find me in Twitter

📚 I write daily for my 112K+ readers on actionable AI developments. Get a 1300+ page Python book as soon as you subscribing (its FREE) ↓↓)

 

Fine-tuning LLM (and YouTube Video Explanations)

Notebook 🟠 YouTube Video
Finetune Llama-3-8B with unsloth 4bit quantized with ORPO
Llama-3 Finetuning on custom dataset with unsloth
CodeLLaMA-34B - Conversational Agent
Inference Yarn-Llama-2-13b-128k with KV Cache to answer quiz on very long textbook
Mistral 7B FineTuning with_PEFT and QLORA
Falcon finetuning on openassistant-guanaco
Fine Tuning Phi 1_5 with PEFT and QLoRA
Web scraping with Large Language Models (LLM)-AnthropicAI + LangChainAI

Fine-tuning LLM

Notebook Colab
📌 Gemma_2b_finetuning_ORPO_full_precision
📌 Jamba_Finetuning_Colab-Pro
📌 Finetune codellama-34B with QLoRA
📌 Mixtral Chatbot with Gradio
📌 togetherai api to run Mixtral
📌 Integrating TogetherAI with LangChain 🦙
📌 Mistral-7B-Instruct_GPTQ - Finetune on finance-alpaca dataset 🦙
📌 Mistral 7b FineTuning with DPO Direct_Preference_Optimization
📌 Finetune llama_2_GPTQ
📌 TinyLlama with Unsloth and_RoPE_Scaling dolly-15 dataset
📌 Tinyllama fine-tuning with Taylor_Swift Song lyrics

LLM Techniques and utils - Explained

LLM Concepts
📌 DPO (Direct Preference Optimization) training and its datasets
📌 4-bit LLM Quantization with GPTQ
📌 Quantize with HF Transformers
📌 Understanding rank r in LoRA and related Matrix_Math
📌 Rotary Embeddings (RopE) is one of the Fundamental Building Blocks of LlaMA-2 Implementation
📌 Chat Templates in HuggingFace
📌 How is Mixtral 8x7B is a dense 47Bn param model
📌 The concept of validation log perplexity in LLM training - a note on fundamentals.
📌 Why we need to identify target_layers for LoRA/QLoRA
📌 Evaluate Token per sec
📌 traversing through nested attributes (or sub-modules) of a PyTorch module
📌 Implementation of Sparse Mixtures-of-Experts layer in PyTorch from Mistral Official Repo
📌 Util method to extract a specific token's representation from the last hidden states of a transformer model.
📌 Convert PyTorch model's parameters and tensors to half-precision floating-point format
📌 Quantizing 🤗 Transformers models with the GPTQ method
📌 Quantize Mixtral-8x7B so it can run in 24GB GPU
📌 What is GGML or GGUF in the world of Large Language Models ?

Other Smaller Language Models


About Collider.club

This card belongs to the curated knowledge base of Collider.club — a closed business club for entrepreneurs, engineers, investors and domain experts building projects for international markets. Members work across DeFi, AI/ML, FinTech, Web3, banking, hardware and venture capital, and the club runs closed sessions on high-margin niches with anonymous speakers.

  • Club: https://collider.club
  • Collection: Collider.club curated card library (mdrss-card/v2)
  • Maintainer: Collider.club editorial team

License

MIT License — Copyright (c) 2026 Collider.club. Full text: LICENSE · https://opensource.org/licenses/MIT

MARKDOWN METRICS
1215words
11headings
74links
0code blocks
MDRSS ASSESSMENT
Scam / risk5/100low
Evidence100/100high confidence
Why MDRSS assigned this score
  • evidence comes from multiple domains
  • some evidence URLs look like primary-source hosts
Evidence (4)
concept:training-fine-tuningorg:collider-club

Discussion 0

Sign in to join the discussion.