0
Back to Roadmap
Phase 1.1 · OptionalOptional — not counted in progress

Transformer & ML Foundations

Neural networks, attention, encoders/decoders, BERT, GPT, KV cache, RoPE, MoE, and quantization — interview-depth ML intuition. Skip if you are focused on building agents, not training models.

21 modules

Estimated read time is shown per module based on lesson length.

Modules

1.Neural Networks~3 min read
2.Gradient Descent~3 min read
3.Backpropagation~3 min read
4.Activation Functions~3 min read
5.Loss Functions~3 min read
6.Optimizers~3 min read
7.Attention~3 min read
8.Self Attention~3 min read
9.Multi Head Attention~4 min read
10.Encoder~3 min read
11.Decoder~3 min read
12.Positional Encoding~3 min read
13.Transformer Architecture~3 min read
14.Word2Vec~3 min read
15.BERT~3 min read
16.GPT~3 min read
17.Inference~3 min read
18.KV Cache~3 min read
19.RoPE~3 min read
20.MoE~3 min read
21.Quantization~3 min read

Navigation

← Generative AI FoundationsLLM Engineering & APIs →