Hello :) Today is Day 62!
A quick summary of today:
- Lecture 8: Self-Attention and Transformers
- Lecture 9: Pretraining
- Lecture 10: Prompting, Reinforcement Learning from Human Feedback
I watched this State of GPT talk by Andrej Karpathy a week or so ago and saw the below slide:
And that was the first time I saw this GPT pipeline. Today, I got to learn about each of these steps. Feels nice knowing how GPT/ChatGPT works.
Anyway ~ my notes are below :)
Lecture 8: Self-Attention and Transformers
Lecture 9: Pretraining
Lecture 10: Prompting, Reinforcement Learning from Human Feedback
Tomorrow we continue diving deeper into the world of NLP.
That is all for today!
See you tomorrow :)