← back to curriculum github ↗

phase 10

Llms From Scratch

24 lessons · click any to read the docs

phase progress

0/ 24 lessons · 0%

Building A Tokenizer

02-building-a-tokenizer

03-data-pipelines

Pre Training Mini Gpt

04-pre-training-mini-gpt

Scaling Distributed

05-scaling-distributed

Instruction Tuning Sft

06-instruction-tuning-sft

Constitutional Ai Self Improvement

09-constitutional-ai-self-improvement

11-quantization

Inference Optimization

12-inference-optimization

Building Complete Llm Pipeline

13-building-complete-llm-pipeline

Open Models Architecture Walkthroughs

14-open-models-architecture-walkthroughs

Speculative Decoding Eagle3

15-speculative-decoding-eagle3

Differential Attention V2

16-differential-attention-v2

Native Sparse Attention

17-native-sparse-attention

Multi Token Prediction

18-multi-token-prediction

Dualpipe Parallelism

19-dualpipe-parallelism

Deepseek V3 Walkthrough

20-deepseek-v3-walkthrough

Jamba Hybrid Ssm Transformer

21-jamba-hybrid-ssm-transformer

Async Hogwild Inference

22-async-hogwild-inference

Speculative Decoding

25-speculative-decoding

Gradient Checkpointing

34-gradient-checkpointing