MDRSS/science/methods/card #1419

🧠 What is Pyre Code?

Card addresshttps://mdrss.com/science/methods/1419
Canonical Markdownhttps://mdrss.com/science/methods/1419/1419.md

Snapshot 2026-08-03 23:56:39 UTC Β· version 1

● published
INDEXABLE MARKDOWN SNAPSHOT

Research document

Open canonical .md

English | δΈ­ζ–‡

πŸ”₯ Pyre Code

Implement the internals of modern AI systems from scratch β€” Transformers, vLLM, TRL, and beyond.

Read the paper, then write the code. No GPU required.

🧠 What is Pyre Code?

76 problems. You write the implementation, a local grading service runs the tests, you see what broke. That's it.

The problems cover what's actually inside Transformers, vLLM, TRL, diffusion models, and GNNs β€” attention variants, training tricks, inference kernels, alignment algorithms, graph neural networks. No GPU needed.

Who is this for?

  • Preparing for ML interviews β€” practice implementing core components under test, not just reading about them
  • Learning by building β€” if you learn best by writing code rather than watching lectures, this is your gym
  • Deepening your understanding β€” you've used nn.MultiheadAttention, now write it yourself

Features

  • Browser editor β€” Monaco with Python syntax highlighting, no IDE setup
  • Instant feedback β€” submit and see pass/fail per test case in seconds
  • Reference solutions β€” compare after your own attempt
  • Progress tracking β€” solved count and attempt history, persisted across sessions
  • AI Help β€” optional AI-powered hints via any OpenAI-compatible API (configure in .env or per-user in the UI)
  • Fully local β€” nothing leaves your machine (unless you opt into AI Help)

Tech Stack

Layer Technology
Frontend Next.js + Monaco Editor + Tailwind CSS
Backend FastAPI grading service
Judge Engine torch_judge β€” executes and validates submissions
Storage SQLite (progress tracking)

πŸ“’ News

  • [2026/04/20] New GNN learning path β€” 8 problems covering GCN, GAT, GIN, MPNN, GraphSAGE, link prediction, and graph autoencoders. πŸ”₯
  • [2026/04/20] New UI redesign with OKLch color system, dark mode, and Geist typography β€” classic design still available via toggle. πŸ”₯
  • [2026/04/13] Submission history β€” review all your past attempts per problem.
  • [2026/04/10] AI Help β€” optional AI-powered hints via any OpenAI-compatible API. πŸ”₯
  • [2026/04/10] Print output capture β€” print() statements now show in test results.
  • [2026/04/09] 68 problems covering Transformers, vLLM, TRL, diffusion models, and more. πŸ”₯
  • [2026/04/09] Initial release of Pyre Code πŸŽ‰

πŸš€ Getting Started

Prerequisites

  • Python 3.11+
  • Node.js 18+

Installation

Option A β€” one-liner (recommended)

macOS / Linux:

git clone https://github.com/whwangovo/pyre-code.git
cd pyre-code
./setup.sh
npm run dev

Windows (PowerShell):

git clone https://github.com/whwangovo/pyre-code.git
cd pyre-code
.\setup.ps1
npm run dev

Windows (CMD):

git clone https://github.com/whwangovo/pyre-code.git
cd pyre-code
setup.bat
npm run dev

The setup script automatically creates a .venv Python environment (prefers uv, falls back to python -m venv), installs all dependencies, then prints the start command.

When .venv exists, npm run dev prefers that project-local Python automatically. If .venv is missing, it falls back to the current shell's python.

Option B β€” conda

git clone https://github.com/whwangovo/pyre-code.git
cd pyre-code
conda create -n pyre python=3.11 -y && conda activate pyre
pip install -e ".[dev]"
npm install
npm run dev   # run with conda env activated

Option C β€” manual (venv)

git clone https://github.com/whwangovo/pyre-code.git
cd pyre-code

# create a Python env β€” pick one:
uv venv --python 3.11 .venv && source .venv/bin/activate && uv pip install -e ".[dev]"
# or: python3 -m venv .venv && source .venv/bin/activate && pip install -e ".[dev]"
# Windows: python -m venv .venv && .venv\Scripts\activate && pip install -e ".[dev]"

npm install
npm run dev

Either way, once running:

  • Grading service β†’ http://localhost:8000
  • Web app β†’ http://localhost:3000

Option D β€” Docker

git clone https://github.com/whwangovo/pyre-code.git
cd pyre-code
docker compose up --build

Open http://localhost:3000. Progress is persisted in a Docker volume. Run docker compose down -v to reset.

AI Help (optional)

To enable server-side AI hints, copy web/.env.example to web/.env and fill in:

AI_HELP_BASE_URL=https://api.openai.com/v1
AI_HELP_API_KEY=sk-...
AI_HELP_MODEL=gpt-4o-mini

Any OpenAI-compatible endpoint works (OpenAI, Anthropic via proxy, Ollama, etc.). Users can also configure their own API key in the UI if no server-side config is set.

πŸ“‹ Problem Set

76 problems organized by category:

Category Problems
Fundamentals ReLU, Softmax, GELU, SwiGLU, Dropout, Embedding, Linear, Kaiming Init, Linear Regression
Normalization LayerNorm, BatchNorm, RMSNorm
Attention Scaled Dot-Product, Multi-Head, Causal, Cross, GQA, Sliding Window, Linear, Flash, Differential, MLA
Position Encoding Sinusoidal PE, RoPE, ALiBi, NTK-aware RoPE
Architecture SwiGLU MLP, GPT-2 Block, ViT Patch, ViT Block, Conv2D, Max Pool, Depthwise Conv, MoE, MoE Load Balance
Training Adam, Cosine LR, Gradient Clipping, Gradient Accumulation, Mixed Precision, Activation Checkpointing
Distributed Tensor Parallel, FSDP, Ring Attention
Inference KV Cache, Top-k Sampling, Beam Search, Speculative Decoding, BPE, INT8 Quantization, Paged Attention
Loss & Alignment Cross Entropy, Label Smoothing, Focal Loss, Contrastive Loss, DPO, GRPO, PPO, Reward Model
Diffusion & DiT Noise Schedule, DDIM Step, Flow Matching, adaLN-Zero
Adaptation LoRA, QLoRA
Reasoning MCTS, Multi-Token Prediction
SSM Mamba SSM
Graph Neural Networks GCN, Graph Readout, GAT, GIN, MPNN, GraphSAGE, Link Prediction, Graph Autoencoder

Learning Paths

Pick one based on what you're working toward:

Path Problems Description
Transformer Internals 12 Activations β†’ Normalization β†’ Attention β†’ GPT-2 Block
Attention & Position Encoding 13 Every attention variant + RoPE, ALiBi, NTK-RoPE
Train a GPT from Scratch 15 Embeddings β†’ architecture β†’ loss β†’ optimizer β†’ training tricks
Inference & Distributed Training 9 KV cache, quantization, sampling, tensor parallel, FSDP
Alignment & Agent Reasoning 6 Reward model β†’ DPO β†’ GRPO β†’ PPO β†’ MCTS
Vision Transformer Pipeline 7 Conv β†’ patch embedding β†’ ViT block
Diffusion Models & DiT 5 Noise schedule β†’ DDIM β†’ flow matching β†’ adaLN-Zero
LLM Frontier Architectures 7 GQA, Differential Attention, MLA, MoE, Multi-Token Prediction
Graph Neural Networks 8 GCN β†’ GAT β†’ GIN β†’ MPNN β†’ GraphSAGE β†’ Link Prediction β†’ GAE 
Not sure where to start?

Fundamentals ──→ Transformer Internals ──→ Train a GPT from Scratch
                       β”‚                          β”‚
                       β–Ό                          β–Ό
              Attention & Position       Inference & Distributed
                       β”‚                          β”‚
                       β–Ό                          β–Ό
              LLM Frontier Archs         Alignment & Reasoning
                       β”‚
               β”Œβ”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”
               β–Ό       β–Ό       β–Ό
     Vision Trans.  Diffusion  Graph Neural Networks

βš™οΈ Configuration

Variable Default Description
GRADING_SERVICE_URL http://localhost:8000 Grading service URL
DB_PATH ./data/pyre.db SQLite database for progress tracking

Set in web/.env.local to override.

πŸ“ Project Structure

pyre/
β”œβ”€β”€ web/                  # Next.js frontend
β”‚   β”œβ”€β”€ src/app/          # Pages and API routes
β”‚   β”œβ”€β”€ src/components/   # UI components
β”‚   └── src/lib/          # Utilities, problem data
β”œβ”€β”€ grading_service/      # FastAPI backend
β”œβ”€β”€ torch_judge/          # Judge engine (problem definitions + test runner)
└── package.json          # Dev scripts (runs frontend + backend concurrently)

🀝 Contributing

Contributions are welcome! Here are some ways you can help:

  • Submit a new problem β€” open a PR with the problem definition and test cases in torch_judge/
  • Report a bug β€” open an issue with steps to reproduce
  • Fix a bug β€” fork, fix, and submit a PR
  • Improve docs β€” typos, clarifications, translations

Please open an issue first for larger changes so we can discuss the approach.

⭐ Star History

πŸ™ Acknowledgements

Problem set and judge engine based on TorchCode by duoan, licensed under MIT.

πŸ“„ License

Distributed under the MIT License. See LICENSE for more information.

MARKDOWN METRICS
930words
20headings
9links
9code blocks
MDRSS ASSESSMENT
Evidence46/100high confidence
Why MDRSS assigned this score
  • Production catalog audit 2026-08-04
  • Taxonomy classified from title, annotation, source and Markdown signals
  • Agent usefulness evaluated from structure, procedures, examples, evidence and retrieval value
Evidence (1)

Discussion 0

Sign in to join the discussion.