← curriculum/phase 10 · Llms From Scratch

edit on github ↗

← all lessons in phase

phase 10 · 24 lessons

01Tokenizers
02Building A Tokenizer
03Data Pipelines
04Pre Training Mini Gpt
05Scaling Distributed
06Instruction Tuning Sft
07Rlhf
08Dpo
09Constitutional Ai Self Improvement
10Evaluation
11Quantization
12Inference Optimization
13Building Complete Llm Pipeline
14Open Models Architecture Walkthroughs
15Speculative Decoding Eagle3
16Differential Attention V2
17Native Sparse Attention
18Multi Token Prediction
19Dualpipe Parallelism
20Deepseek V3 Walkthrough
21Jamba Hybrid Ssm Transformer
22Async Hogwild Inference
23Speculative Decoding
24Gradient Checkpointing

‹

›

phase 10 · lesson 17 of 24

Native Sparse Attention

17-native-sparse-attention·full lesson folder ↗

Differential Attention V2

complete & next →

Multi Token Prediction