← back to curriculum github ↗

phase 7

Transformers Deep Dive

16 lessons · click any to read the docs

phase progress

0/ 16 lessons · 0%

Why Transformers

01-why-transformers

Self Attention From Scratch

02-self-attention-from-scratch

Multi Head Attention

03-multi-head-attention

Positional Encoding

04-positional-encoding

Full Transformer

05-full-transformer

Bert Masked Language Modeling

06-bert-masked-language-modeling

Gpt Causal Language Modeling

07-gpt-causal-language-modeling

T5 Bart Encoder Decoder

08-t5-bart-encoder-decoder

Vision Transformers

09-vision-transformers

Audio Transformers Whisper

10-audio-transformers-whisper

Mixture Of Experts

11-mixture-of-experts

Kv Cache Flash Attention

12-kv-cache-flash-attention

13-scaling-laws

Build A Transformer Capstone

14-build-a-transformer-capstone

Attention Variants

15-attention-variants

Speculative Decoding

16-speculative-decoding