学习资源

Compilation of categorized learning resources

PyTorch/LLM Basics

• from scratch

• tudui

• gpt2

• illustrated gpt2

---

AutoRegressive Model

• PixelCNN by Google DeepMind

• PixelCNN++ by OpenAI

---

Cachekit resources recommended by Jing Liu

• vllm

• Nvidia Inference Optimization blog

• Yao Fu

• Yao Fu 2

---

visual generative model

• MIT 6.S978 Deep Generative Models

---

Flash Attention

• Flash Attention

---

Sparse Attention resources recommended by Yefei He

• MInference

• cross attention: online blogs will just be ok

---

Diffusion model resources recommended by bohan

• Diffusion models

• Generative models

---

Vision transformer

• ViT

• CNN(ResNet)

---

Multimodal LLM

• LLaVA

• NExT-GPT

---

Attention Basics

• sebastian

---

Relevant Online Courses

• cs231n

• HanLAB TinyML

• CMU dlsys

• Parallel Computing

HuggingFace resources

• transformer tokenizer

General videos

Terence Tao at IMO 2024

Tokenizer

General advice for PhD

Andrej Karpathy